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Abstract: During the 1990s and 2000s, a policy known as Education with Community Participation 
(EDUCO) not only became the cornerstone of education reform in El Salvador but also became a 
global education policy, one which is known for decentralizing to rural families the responsibility for 
hiring and firing teachers. As is shown in this paper, its rise to fame was not only a product of the 
particular political-economic context in which it was borne, but was also a product of the impact 
evaluations produced by the World Bank, which served as the evidence base through which this and 
other international institutions could legitimately promote the neoliberal model of community 
involvement represented by EDUCO. Problematically, however, a reappraisal of these impact 
evaluations reveals, first, that their findings and conclusions around significant effects were not 
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warranted and, second, that the entire impact evaluation enterprise is fundamentally flawed due to 
the financial-political-intellectual complex out of which these studies emerged and back into which 
they fed as they were used to advocate for market-oriented policy solutions around the world. Thus, 
in addition to explaining the reform dynamics that gave rise to EDUCO, this paper (a) systematically 
reviews the findings and limitations of each of the six impact studies that constitute the international 
knowledge base around this policy, (b) reconsiders what we can reasonably claim to know about 
EDUCO, (c) reflects on the national and international implications of the critical review presented 
here, and (d) remarks on the shortcomings of—and the alternatives to—impact evaluations as a 
means to produce policy-relevant findings. 

Key words: El Salvador; decentralization, EDUCO; World Bank; political economy; knowledge 
production; global education reform; impact evaluations 

El Programa EDUCO, las evaluaciones de impacto y la economla polltica de la 
reforma educativa global 

Resumen: Durante las decadas de 1990 y 2000, una polltica conocida como Education con 
Participacion de la comunidad (EDUCO) se convirtio no solo en la piedra angular de la reforma 
educativa en El Salvador, sino ademas se convirtio en una polltica educativa de caracter global, 
conocida por descentralizar, hacia las familias rurales, la responsabilidad de contratar y despedir 
maestros. Como se presenta en este articulo, su ascenso a la fama no fue producto unicamente del 
contexto polltico-economico en el que nacio; su fama tambien es el resultado de las evaluaciones de 
impacto producidas por el Banco Mundial, que sirvieron como base de evidencia a traves de la cual 
esta y otras instituciones internacionales podian promover, de forma legitima, el modelo neoliberal 
de participacion de la comunidad representado por EDUCO. Sin embargo, de una manera 
problematica, una re-apreciacion de estas evaluaciones de impacto, revela, en primer lugar, que los 
hallazgos y conclusiones sobre los efectos significativos de EDUCO no fueron justificados, y, en 
segundo lugar, que la production de las evaluaciones de impacto en su conjunto es 
fundamentalmente defectuosa debido al nexo intelectual-polltico-financiero del cual emergieron 
estos estudios, y hacia el cual retornaron para promover—alrededor del mundo—soluciones de 
pollticas orientadas al mercado. En este sentido, para explicar las dinamicas de la reforma que dieron 
lugar a EDUCO, este documento (a) revisa sistematicamente los hallazgos y limitantes de cada uno 
de los seis estudios de impacto que constituyen la base international de conocimiento alrededor de 
esta polltica, (b) reconsidera lo que, de forma razonable, podemos exigir saber sobre EDUCO, (c) 
refleja las implicaciones nacionales e internacionales de la revision critica aqui presentada (d) resalta 
las falencias del metodo de investigation de las evaluaciones de impacto y (e) sugiere varios metodos 
alternatives que pueden ofrecer hallazgos relevantes para las pollticas. 

Palabras clave: El Salvador; descentralizacion; EDUCO; Banco Mundial; economla polltica; 
production de conocimiento; reforma educativa global; evaluaciones de impacto 

O Programa EDUCO, as avalia§oes de impacto, a economia polltica da reforma educativa 
global 

Resumo: Durante as decadas de 1990 e 2000, uma polltica conhecida como Educa^ao com 
Participa^ao Comunitaria (EDUCO) tornou-se nao so a pedra angular da reforma da educa^ao em 
El Salvador, mas tambem se tornou uma polltica educational global conhecida por descentralizar 
para as familias rurais, a responsabilidade de contratar e demitir professores. Como mostra este 
artigo, sua ascensao a fama nao foi o resultado so do contexto politico e economico em que nasceu; 
sua fama e tambem o resultado das avalia^oes de impacto produzidas pelo Banco Mundial, que 
serviu como base de evidencias para que esta e outras institui^Ses internacionais pudessem 
promover e legitimar o modelo neoliberal de envolvimento da comunidade representada por 
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EDUCO. No entanto, de maneira problematica, uma reavaliagao dessas avalia^Ses de impacto, 
revela, em primeiro lugar, que os resultados e conclusoes sobre efeitos signiflcativos de EDUCO 
nao foram justiflcadas e, por outro, que a produgao do avalia^oes de impacto como um todo e 
fundamentalmente falida devido ao nexo financeiro-politico-intelectual, que surgiram a partir desses 
estudos, e a qual retornou para promover solugoes mundiais politicas orientadas para o mercado. 
Neste sentido, para explicar a dinamica da reforma que levaram a EDUCO, este documento (a) 
analisa sistematicamente os resultados e as limita^Ses de cada um dos seis estudos de impacto que 
constituem a base de conhecimento internacional sobre esta politica, (b) reconsidera o que, 
razoavelmente, podemos saber sobre EDUCO, (c) reflete sobre as implicates nacionais e 
internacionais de revisao critica aqui apresentada (d) destaca as deficiencias do metodo de avalia^ao 
de impacto e de pesquisa (e) ele sugere varios metodos alternatives que podem oferecer importantes 
descobertas para a politica. 

Palavras-chave: El Salvador; descentralizacao; EDUCO, Banco Mundial; economia politica; 
produ^ao de conhecimento; reforma da educa^ao mundial 


The EDUCO Program, Impact Evaluations and the Political Economy of 

Global Education Reform 1 


During the 1990s and 2000s, a policy known as Education with Community Participation 
(EDUCO) not only became the cornerstone of education reform in El Salvador but also became a 
global education policy (Edwards, forthcoming), in that it circulated around the world in the spaces 
where the international agenda for education reform is created and communicated (Verger, Novelli, 
& Kosar-Altinyelken, 2012). Indeed, since the mid-1990s, this policy, which decentralized the 
responsibility for hiring and firing teachers to parents in mral communities, has been studied by 
technical groups from at least 18 countries (Meza, Guzman & de Varela, 2004a), 2 in addition to 
being promoted by numerous international organizations through high profile publications, 
including some of the most influential reports in the realm of international development (Edwards, 
2013b). A select few key examples are: 

• The World Bank’s World Development Report 2004, which focused on making public 
services more accountable (World Bank, 2003). 

• The report in 2005 of the United Nations task force on gender equity (Birdsall, Levine & 
Ibrahim, 2005). 

• A publication by the Brookings Institute (an influential think tank in Washington DC) in 
2008 on effective development practices (Hartman & Linn, 2008). 

• The Global Monitoring Reports by UNESCO in both 2009 and 2011, which addressed 
education governance and education in conflict-affected contexts (UNESCO, 2008, 2011). 

While many political and economic factors influenced its emergence and subsequent trajectory 
(Edwards, forthcoming), one important issue that previous studies of this program have tended not 


1 This paper is partially based on a presentation with the title “The EDUCO Studies: Between Fiction and 
Reality,” given at the Universidad Centroamericana on March 26, 2014. 

2 The list of countries includes at least Guatemala, Honduras, Colombia, Chile, Nicaragua, Panama, 
Dominican Republic, Mexico, Brazil, Paraguay, Bolivia, Ghana, Mauritius Islands, Madagascar, Chad, 
Morocco, Senegal and Mauritania (Meza, Guzman & de Varela, 2004). 
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to highlight is the central role played by research and impact evaluations. The present manuscript 
attempts to fill this gap in the literature. It does this, first, by presenting a critical review of the 
evidence base that has been used to promote EDUCO, and, then, by discussing the meaning of that 
evidence—in light of the reappraisal presented here—within the political economic context that 
produced EDUCO and the impact evaluations that legitimated its dissemination and promotion 
globally. 

Going forward, this article has eight sections. The first establishes the predominance of 
international organizations in education reform, with a focus on the World Bank, its policy 
preferences and the ways that impact evaluations help to create a demand for the policy solutions it 
offers. The second section then defines what is meant by a critical analysis of impact evaluations 
within the political economy of global education reform. The third section explains the political- 
economic context in El Salvador at the end of the 1980s and beginning of the 1990s. This period is 
the starting point because the dynamics which characterize it influenced the emergence of EDUCO 
in 1991 in important ways. The fourth section then shows how the strategic activity of key actors 
intersected with contextual constrains and then crystallized around certain policy priorities that 
favored EDUCO. Having clarified stmctural constraints and the vested interests in the second 
section, the fifth section systematically reviews the findings and limitations of six econometric 
studies that were integral to EDUCO’s success and which have contributed to EDUCO’s high- 
profile nature in the international realm. The sixth section contains additional discussion, both on 
how these studies should be understood as well as about what we can reasonably claim to know 
about EDUCO from a critical perspective. The seventh section considers the national and 
international implications of the critical review presented here, particularly as they relate to the 
political-economic dynamics of the education reform process that encompassed EDUCO’s 
emergence and trajectory. The eighth section offers substantive concluding remarks on the 
shortcomings of—and the alternatives to—impact evaluations, particularly given the financial- 
political-intellectual complex out of which these studies tend to emerge and back into which they 
tend to feed as they are frequently used to advocate for market-oriented policy solutions in periods 
of system reform. 


The World Bank, Global Education Reform, and Knowledge Production 


In the literature on education reform in low-income countries, it has long been 
acknowledged that international organizations are influential (Berman, 1979, 1992), with perhaps no 
other institution receiving more attention than the World Bank (see, e.g., Collins & Wiseman, 2012; 
Klees, Samoff, & Stromquist, 2012). Moreover, as has been acknowledged, this influence manifests 
or is exercised in various ways (Samoff, 2009), ways that demonstrate both the materialist and 
semiotic forms of power. While the former (i.e., materialist form) has arguably tended to receive 
more attention from scholars (e.g., when they focus on structural adjustment programs and the ways 
that the World Bank has coerced countries into enacting reforms by threatening to withhold loan 
funding otherwise), it can be said that these forms of power are inter-twined and that the latter is 
just as important. Indeed, part of the reason the World Bank is “the largest development research 
institution in the world” is precisely because of its financial resources (Gilbert & Vines, 2006, p. 49). 
And one implication of the fact that the World Bank operates in this way as a “knowledge 
institution” is that it can influence which reforms are seen as legitimate and desirable, through its 
ability to elevate and promote its research in strategic ways within the global governance of 
education (Verger, 2012). Interestingly, recent research has shown that actors in the field of global 
education policy have come to rely on the World Bank’s research, to the point where they look for it 
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and feel uncomfortable without it, even in spaces of consultation designed to be open to other 
perspectives (Verger, Edwards, & Kosar-Altinyelken, 2014). 

Of course, an important aspect of the production of knowledge for development (like other 
forms of knowledge) is that it is not free from bias (Cox, 2002) and that its valuation is not free 
from the prevailing norms or standards of quality (Walters, Lareau, & Ranis, 2009). Scholars such as 
Broad (2006) and Klees and Edwards (2014) have addressed this issue in relation to the World Bank, 
pointing out that this institution, at its core, is guided by a neoliberal worldview and that the research 
it produces reflects that worldview by promoting policies and reforms based in market or quasi¬ 
market principles. That is, while the World Bank frames its findings and the policies it recommends 
as the product of scientific inquiry guided by the highest standards of social science (see, e.g., 
Khandker, Kookwal, & Samad, 2010; Verger, 2012), these claims are undercut by the fact that the 
general nature of these findings and the recommendations that follow are predetermined. While the 
coefficients might not be known a priori, we can be sure that the World Bank, in the realm of 
education reform, is going to demonstrate and emphasize the necessity of instituting policies for 
school competition, parental choice, teacher accountability, and, more recently, low-fee private 
schools, to mention a few examples. 

Elowever, in order to sustain its business model, the World Bank, like other international 
development banks, must be able to demonstrate that its policies are successful. Consequently, the 
World Bank not only promotes reforms that are—according to contemporary wisdom in the current 
context of neoliberalism (Spring, 2015)—accepted as appropriate to enhance such outcomes as 
quality and efficiency, but this organization also seeks out and conducts its own research on 
promising pilot programs or on-going reforms, even if it is also the World Bank that is financing 
those examples, despite the conflict of interest. If it can be shown through impact evaluations that a 
particular policy causes positive outcomes in terms of student achievement or student dropout, for 
example, then that policy will, on the basis of those evaluations, be promoted globally through 
World Bank publications and will be included in the short list of acceptable reforms that the World 
Bank is willing to support in other countries (Heyneman, 2003; Steiner-Khamsi, 2012). This state of 
affairs leads to the observation that the World Bank produces “policy-based evidence” as opposed 
to “evidence-based policy.” The way this happens is a primary focus of the present article. 


Critically Analyzing Impact Evaluations within the Political Economy of 

Global Education Reform 


A critical analysis of impact evaluations within the political economy of global education 
reform means assessing a set of evaluative studies that have been carried out on a policy or program 
not only in relation to the methodological limitations of the studies but also in relation to the 
meaning or significance of those studies for policy, for practice, and for the vested interests that may 
be advanced (or not) on a global scale from their production and the findings they contain. Realizing 
this kind of review clearly means going beyond (a) a consideration of the stated results and 
methodological limitations of a corpus of studies, though this is an important step (as, e.g., in Bryk, 
1981); it also means (b) understanding those studies within the multi-level (i.e., international, 
national, local) political-financial-intellectual complex out of which they emerged, (c) appraising the 
implications of those studies (in terms of their stated findings but also in terms of their political 
meaning) for the vested actors and institutions that facilitated, utilized, and/or otherwise benefitted 
from the studies; and (d) tending to the ways that the studies contribute to oppressing certain 
perspectives or policy preferences while elevating others at each level from the local to the global 
(Klees & Edwards, 2014; Lubienski, Weitzel & Lubienski, 2009; Robertson & Dale, 2009; Samoff, 
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1993, 2012; Verger, 2012). The present article engages with these steps in relation to the series of 
impact evaluations that were produced by the World Bank of the EDUCO program. 

It is suggested that, ultimately, the insights gained through such an analysis are more 
significant than asking whether a program worked or whether a study’s findings hold up to a critical 
review of their methodological shortcomings alone. The added benefit of taking a critical review of 
empirical findings (based, e.g., on their methodological basis) and placing those findings within the 
political economy of education reform and the political economy of research production is that it 
shifts our gaze away from an exclusive focus on research findings as if they were the product of a 
clinical and objective process to instead focus on how those findings may reproduce existing power 
structures and/or advance certain policy preferences over others; or, put differently, we are 
encouraged to look anew at how and why research findings, by virtue of how they are produced and 
the power relations that govern their production, may disadvantage certain groups, and with what 
implications (Water, 1998). Arguably, carrying out such reviews is becoming more imperative, 
particularly as the relationships among the policymakers, funders, evaluators, publishers, and 
promoters of research and “policy-relevant” findings continue to evolve and to blur on a global 
scale and in ways that can involve a conflict of interest wherein those who fund and carry out 
research are often also the beneficiaries of that research (Ball, 2009, 2010; Goldie, Linick, Jabbar, & 
Lubienski, 2014; Lubienski, Scott, & DeBray, 2014; Nambissan & Ball, 2010; Scott & Jabbar, 2014; 
Verger, 2012). 

In practice, then, in order to follow through with the type of critical analysis described 
above, one must move from the macro to the micro and then from the micro back to the macro. 
That is to say, in going from the macro to the micro, one must understand: (a) the historical- 
structural aspects of the context in question and, in particular, how they constrain the actors in that 
context; (b) the range of involved actors and the ways they are situated within the larger context, as 
well as their policy preferences; (c) the relevant processes or opportunities for policy change that 
emerge, in addition to the rules, requirements, and expectations that govern those processes; (d) the 
production and utilization of “legitimate” and “policy-relevant” knowledge (in this case, impact 
evaluations), including who generated and funded that knowledge; and (e) the findings of the 
relevant studies not only as stated by their authors but also in terms of what one can reasonably 
claim to know after considering the methodological and practical limitations that accompany the 
production of the studies in question. At this point, one not only has an understanding of the 
empirical findings of a body of research but also has a grasp of the political significance of that 
research and the ways that it has been employed. Subsequently, in going from the micro to the 
macro, one begins by (f) clearly problematizing that research in technical terms and then considering 
the implications of both the stated findings and their critical reinterpretation (that is, what one can 
reasonably claim to know in light of their limitations) for a number of issues, including: (g) processes 
of policymaking or educational reform, (h) the marginalization of certain political or alternative 
perspectives, (i) the advancement of political and institutional interests, (j) the emergence or 
prevalence of global reform trends, and, at the most broad level, (k) the maintenance or extension of 
the prevailing ideology or common sense in a certain area (in this case, educational governance). 
While other authors have arguably employed this approach when analyzing research findings (e.g., 
Edwards, 2012; Klees & Wells, 1983; Lubienski, Weitzel, & Lubienski, 2009; Samoff, 1993; Verger, 
2012), it is hoped that specifying the steps in the process of analysis will make the process more 
clear and more accessible for scholars to utilize. 

The above discussion thus indicates that the analysis presented in this paper is based on 
much more than a review of impact evaluations themselves. Indeed, the findings discussed here 
have resulted from on-going research that began in 2009 (see, e.g., Edwards, 2012, 2013a, 2013b, 
2015; Edwards & DeMatthews, 2014; Edwards & Klees, 2012; Edwards, Victoria, & Martin, 2015). 
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To summarize, in addition to extensively reviewing the literature on EDUCO, the present piece is 
based on over 100 interviews with key actors from the Ministry of Education in El Salvador 
(MINED); the United States Agency for International Development (USAID); the World Bank; the 
United Nations Education, Science, and Culture Organization (UNESCO); and the Central 
American University, not to mention other organizations such as think tanks within and outside El 
Salvador as well as national and international non-governmental organizations. For data collection, 
the author also reviewed the archival collections of the organizations mentioned above (with the 
content of those collections covering the years of the 1970s to 2012) and participated as an observer 
in events related to the global promotion of EDUCO and its evidence base. Ten months of full-time 
fieldwork in El Salvador during 2012-2013 through a Fulbright Research Grant helped to ensure 
that data collection for this study was both broad and deep. On the basis of this information, it was 
possible to engage in the multi-step analytic process detailed in the previous paragraph. 


Historical Context and Structural Transformation in El Salvador Prior to 

EDUCO 


To understand the emergence of EDUCO it is necessary to first grasp the political economic 
context of the 1980s in El Salvador. During this decade, El Salvador found itself mired in an on¬ 
going civil war which had, on one side, the government, the landed oligarchy, and the armed forces 
and, on the other, a coalition of five rebel groups fighting for socialist reform of the land, economy, 
and social services (LeoGrande, 1998). Collectively, these rebel groups were known as the 
Farabundo Marti National Liberation Front (FMLN). 3 The impetus for the uprising of these five 
groups was the continual repression and persecution by the government and its death squads of the 
population, particularly in rural areas, which was increasingly unaccepting of the country’s extreme 
inequality and its concentration of land ownership in the hands of a few (Montgomery, 1995). 
During 1980-1984 alone an estimated 70,000 civilians were murdered at the hands of the death 
squads (Meade, 2010; Orr, 2001). 

Layered upon this state of affairs were three additional—and intertwined—aspects of the 
political economic context. The first was the involvement of the United States. With the Cold War 
as a backdrop, the socialist uprising in El Salvador (just as in neighboring Nicaragua) was an issue of 
particular concern. In response to FMLN offensives, the United States provided significant 
infusions of military and social aid. To put into perspective the magnitude of the aid provided, 
consider that, while the United States provided approximately $150 million in aid to El Salvador 
from 1963 to 1979, it jumped to approximately $6 billion between 1980 and 1992 (Robinson, 2003, 
p. 89). In 1985, military aid alone amounted to $533 million (Booth, Wade, & Walker, 2006, p. 105). 

The second aspect to note was the international promotion of neoliberal ideology and 
reform by the Reagan administration internationally and by USAID within El Salvador. On this 
latter point, USAID perceived that the context of the civil war provided an opportunity to 
reconfigure the political landscape of the country in order to promote the adoption of its preferred 
economic and social policies (Robinson, 2003). In particular, USAID was interested to counter the 
policy orientation of President Jose Napoleon Duarte (1984-1989), a populist who did not favor 
liberalizing the economy, but whom the U.S. Congress felt forced to support to ensure stability of 
the country during the civil war (LeoGrande, 1998). 


3 These five political-military organizations were known as the Popular Forces of Liberation, National 
Resistance, Revolutionary Army of the People, Communist Party of El Salvador, and Revolutionary Party of 
Central American Workers (Montgomery, 1995). 
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In line with this strategy, USAID supported the creation of a think tank, the Salvadoran 
Foundation for Economic and Social Development (FUSADES, for its name in Spanish). This 
organization served as a key space through which to (a) host famous international economists, such 
as the “Chicago Boys” (Klein, 2007); (b) lend credibility to neoliberal reform principles; and (c) raise 
the profile of Salvadoran economists and businessmen who shared USAID’s policy preferences 
(discussed further below). That USAID provided FUSADES with $150 million in contracts during 
1983-1993 shows the importance of the latter to the former. 

In 1986, USAID also facilitated the creation of the Business Foundation for Educational 
Development (FEPADE, for its name in Spanish). As a technocratic organization, FEPADE 
diagnosed the needs of the education sector and worked on USAID’s education projects. It also 
provided training and capacity building to the MINED. Given its origins and orientation, FEPADE 
was a key source of technical capacity for USAID and the Salvadoran government, particularly 
during the 1990s, once the rightwing party assumed power. It is of note that the minister of 
education during 1990-1998—Cecilia Gallardo de Cano—was an education specialist at FEPADE 
before becoming the Minister of Education. Crucially, she also came from the core of the ARENA 
party and, as such, was closely networked with members from its highest echelon, including 
President Cristiani. Moreover, she was keen political player who aspired to become president one 
day. 

Before proceeding with the developments of the 1990s, it is first important to highlight the 
third aspect of the political-economic context. This is that the eventual success (in 1989) of 
rightwing party known as the National Republican Alliance (ARENA) was due in part to the failure 
of the Christian Democrat party under the leadership of Duarte. Initially, after his election in 1984, 
Duarte was handicapped by the rightist majority in the National Assembly, and so could not move 
forward with the agrarian reform that had been initiated by the civilian junta that was briefly in 
power prior to 1984. 4 After 1985, even with control of the National Assembly, Duarte and his party 
were restricted by the military, USAID, and the conservative and reform-minded portion of the 
Salvadoran business elite. In the face of resistance by these actors, Duarte could neither bring about 
peace nor institute social welfare programs, as he had hoped to do. Duarte also failed to open the 
economy the way that many Salvadoran businessmen and the Reagan administration would have 
liked. Instead, he implemented economic reforms that ultimately accelerated inflation and imposed 
higher taxes on imports (LeoGrande, 1998). 

Subsequently, and with support from USAID, the rightwing candidate for president, Alfredo 
Cristiani, won the presidency in 1989. Not surprisingly, Cristiani and “at least 17 business leaders 
and persons linked with ... [FUSADES] became part of the new government,” including the 
ministers of planning and finance and the head of the Central Reserve Bank (Segovia, 1996, p. 55; 
Eriksson, Kremer & Margaret, 2000). Thereafter, they pursued the economic reform package that 
was being promoted by FUSADES and USAID. Though the economic and public-sector reforms 
enacted were often grouped under the innocuous label of “modernization,” it was clear that they 
represented the implementation of a neoliberal perspective on policy. To that end, as documents 
from the World Bank (1991—from dissertation) indicate, in the month following the assumption of 
power, Cristiani “initiated a major stabilization and stmctural adjustment program” in order to 
“create a more liberalized, private-sector led economy” (p. 5). The first stmctural adjustment loan 


4 See Montgomery (1995, ppp. 137-138, 187) for more on the agrarian reform. In short, this reform, which 
had first begun in 1980, sought to nationalize large farms (over 100 hectares) and to transfer ownership of 
land to those who rented it. In early 1985, the budget for this program was eliminated by the National 
Assembly. 
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from the World Bank would be approved in 1991, followed by a second one in 1993. 5 After his 
election, Cristiani also began to call for peace talks with the FMLN. At least in part, this was due to 
the fact Cristiani and other business elite realized that their investments in banking and industry 
would continue to suffer in the context of ongoing conflict. Peace Accords were finally signed in 
January 1992, with the help of the United Nations. 


The Emergence of EDUCO: Process Dynamics and Vested Interests 


A first priority of the Cristiani administration was to address the state and management of 
the education system. In this area, the immediate challenge for the new Minister of Education, 

Cecilia Gallardo de Cano, who assumed her post in 1990, was to identify an acceptable policy to 
expand educational access. Countrywide, by 1990, at least 37% of children ages 7-14 were out of 
school, and in the conflict-affected areas government-provided education services were non-existent 
(MINED, 1990). Importantly, however, in searching for reform options, Gallardo had to take into 
account three constraints. First, the reform strategy had to fit with the larger focus on 
“modernization” that was a core characteristic of the Cristiani government, as already mentioned. 
For public services like education, this meant reducing bureaucracy and introducing new service 
provision arrangements that would ensure efficiency, effectiveness and accountability. Second, 
Gallardo had to circumvent the teachers’ unions, who were not only anti-privatization but also allies 
of the FMLN. Third, the government saw it as necessary to incorporate the communities and 
schools linked with the FMLN into the official education system. 6 This third condition was 
particularly essential from the government’s perspective because the civil war had not yet concluded 
at the time that Gallardo was searching for reform options, and thus the government was still 
looking for ways to subjugate the regions and schools controlled by the FMLN. 

While the constraints specified in the previous paragraph and the preceding section flowed 
from the structural characteristics of the political-economic context of the 1980s, going forward, 
during 1990-1994, these constraints would, first, combine with the strategic action of key actors and, 
then, through that interaction, would lead to the entrenchment of policy priorities. Importantly for 
our purposes, the way in which vested interests crystallized around EDUCO in this period of 
transition ensured that EDUCO became a political priority for multiple actors—and political priority 
that would be furthered by the elaboration of impact studies during 1994-2005. The evolution of the 
period 1990-1994 can be distilled to three “moments.” 

In the first moment, in mid-year 1990, a consultant from UNESCO was tasked with 
conducting an education sector assessment in order to understand the state of education in each of 
the country’s 14 departments. This was a necessary task given that representatives of the 
government had not visited many rural parts of the country for 10 years due to the on-going 
conflict. By conducing this assessment, the consultant realized a reform model based in community 
management of education could be feasible because she observed first-hand that many communities 
(but especially in conflict-affected areas and in areas controlled by the FMLN) were already engaging 
in self-management of the education of their kids due to the lack of government-provided services 
(Gillies, 2010; MINED, 1990). In these areas, a community member offered to teach basic education 
in exchange for minimal, voluntary contributions (e.g., nominal payment or food donations). 


5 These were followed with a “public sector modernization” loan from the World Bank in 1996. For more on 
the significance of these loans in the context of total aid during the 1990s, see Rosa and Foley (2000). 

6 There were approximately 1000 popular education teachers from the FMLN (Alvear Galindo, 2002, p. 189, 

200 ). 
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Moreover, in FMLN areas, the schooling provided was grounded in the tradition of liberation 
theology and in the teachings of Paulo Freire in that it taught students to identify and to mobilize 
against the political-economic stmctures that ensured their oppressed position. For short, this 
approach is known as “popular education” (Hammond, 1998). But the larger point is that the 
Salvadoran government was threatened by the educational approach of the FMLN, as well as by the 
enduring low-level warfare. 

The second moment spanned late 1990 to mid-1991. During this time, the suggestion from 
the UNESCO consultant was at first received with skepticism. The Minister of Education did not 
initially see this idea as viable in political terms because it was based on the management practices of 
FMLN-affiliated communities. For its part, the World Bank doubted that it would be possible to 
implement a community-management model when the majority of parents had low levels of literacy. 
Elowever, after successfully testing this model in a pilot program—and thanks to the persistence of 
the UNESCO consultant—both the Minister of Education and the World Bank began to realize the 
opportunity that such a model represented. That is, it occurred to the Minister of Education of 
Education that this model met her constraints because it was innovative (i.e., it helped to 
“modernize” the MINED by dealing a blow to the central bureaucracy, which had traditionally been 
responsible to teacher management) and because it would allow her to incorporate and, thereby, to 
subjugate the FMLN communities. In addition, both the Minister of Education and the World Bank 
both saw a community model as a way to weaken the teachers unions because it would be possible 
to require that the teachers in this new program work on one-year contracts that would be 
renewable at the discretion of the community. And since these contracts would only affect those 
teachers working in this new program, the teachers unions did not put up sufficient resistance to 
block this reform model, as they had done with previously when the Minister of Education explored 
the possibility of adopting the voucher model from Chile (Edwards, forthcoming). 

Separately, it dawned on the World Bank representatives working in El Salvador that this 
community-based model—if it were successful—would be a bankable policy in that they could sell it 
to other countries, especially since the international reform trends of the day heavily favored efforts 
towards the decentralization of central government functions and the “participation” of stakeholders 
in service provision (Edwards, 2012; Edwards & DeMatthews, 2014). But before moving ahead, two 
points are necessary here to illuminate the pressures and trends of the time. First, during the 1980s, 
and continuing into the 1990s, the World Bank other international organizations were preoccupied 
with the idea of decentralization (Edwards, 2012; Montero & Samuels, 2004), which was a popular 
idea in international development during the 1980s (Conyers, 1983). In the first of these decades, the 
World Bank had conceptual models that predicted increased efficiency and enhanced effectiveness 
from the decentralization of education systems (Winkler, 1989), though they did not have empirical 
evidence. Moreover, before the EDUCO program, it had not occurred to the World Bank that it 
would be possible to decentralize to the community level; they were instead focused on municipal or 
departmental levels. Second, internationally, in the late 1980s and early 1990s, much attention was 
drawn by specialists in in international development to the idea that “participation” was a central 
component of “good” development strategies (Cornwall, 2006; Leal, 2007), especially when it came 
to education policy (Edwards, 2012). This was reflected in March 1990 by the high-profile 
declaration of the World Conference on Education for All (WCEFA), which asserted that 
“partnerships at the community level ... should be encouraged” because “they can help harmonize 
activities, utilize resources more effectively, and mobilize additional financial and human resources” 
(WCEFA Secretariat 1990, p. 58, as cited in Bray, 2003, p. 32). During this time, international 
organizations also highlighted the concept of participation. In particular, USAID raised the issue as a 
means to promote democracy and the effectiveness of public institutions (Montero & Samuels, 
2004). The World Bank, in addition to emphasizing the issue of effectiveness, honed in as well on 
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the potential of decentralization to result in more efficient and accountable service provision 
arrangements (Edwards, 2012). 

For these reasons, by mid-1991, the Minister of Education and the World Bank decided to 
place the full weight of their support behind this model. This was a major turning point because it 
was impossible for any education reform to be successful without the backing of the World Bank— 
the institution that would finance the overhaul of the education sector’s governance system. Thus, in 
May 1991, these actors agreed on a first loan for the education sector that included $10.3 million to 
solidify and extend the community management model, which, from then on, was known as 
“Education with Community Participation,” or EDUCO, as previously noted. Over time, this 
program received support from the Inter-American Development Bank as well as from the World 
Bank. Indeed, these two institutions issued a total of $69.3 million in loans to ingrain, to scale up, 
and to improve the EDUCO program during 1991-2006. It is also notable that the decision to 
pursue this reform occurred well before the Peace Accords of January 1992, with a key implication 
being that this education governance reform strategy was selected before the war had ended and, 
concomitantly, before the official (i.e., public and democratic) post-conflict process of research and 
consultation had a chance to begin (Edwards, Victoria, & Martin, 2015). 

The third moment extended from mid-1991 to 1994. In this period, through extensive and 
intensive World Bank technical assistance, the features of this program were refined. Not 
surprisingly, both the theory that guided the program’s development and the practical arrangements 
that gave life to the program were guided by a neoliberal perspective on education management. To 
be specific, the key features of the program were as follows: a group of five parents from each 
community was elected to form a Community Education Association (Asociacion Comunal para la 
Education, ACE), an entity which was given legal status and which was charged with contracting 
and paying (with funds transferred from the MINED to the ACE via bank account) the teacher(s) 
that would work in their schools. These ACEs were also thought to act as accountability levers that 
reduced teacher absenteeism and resulted in increased teacher effort in the classroom, especially 
since teachers were hired on one-year contracts. With the funds provided by the MINED, each 
ACE additionally had the responsibility of purchasing necessary didactic materials. These 
arrangements were assumed to reduce waste (thereby increasing the efficiency of provision) and to 
enhance student achievement (thereby increasing effectiveness). Given this arrangement, parents in 
disadvantaged mral communities were being asked to assume responsibilities traditionally borne by 
the State, and without compensation. In addition, community members were often expected to 
provide the time, labor, and materials either to constmct or maintain their local school, particularly 
at the beginning of the program—and this was in addition to the informal school fees they already 
incurred to cover basic costs associated with schooling. The MINED, on the other hand, was 
responsible for the following: facilitating the creation of the ACEs; training the members of the 
ACEs in administrative and accounting procedures; setting the minimum criteria for teacher 
selection by the ACEs; designing and providing curricula; and overall coordination, supervision, and 
monitoring of the program as it was scaled up (Reimers, 1997; World Bank, 1994). 8 

Thus, from late 1991 onwards, the Minister was committed to scaling up and widely 
implementing the EDUCO program, which soon became the only program through which the 


7 While the EDUCO program was initially only intended as a strategy to provide education at the preschool 
level and in grades 1-3, it was subsequently expanded in 1994 to cover through grade 6 and then again in 1997 
to cover through grade 9 (Meza, Guzman, & Varela, 2004). After 2005, even some high schools became 
EDUCO schools (Gilles, Crouch, & Florez, 2010). 

8 For additional details on the technical aspects of the EDUCO program, see Gilles, Crouch and Florez 

( 2010 ). 
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Minister would allow educational access to be augmented. By 1993, the program was operating in all 
14 departments of the country, and by 1995 the program had been institutionalized in the sense that 
the EDUCO office had been incorporated into the larger administrative structure of the MINED 
(Edwards, 2013b). In statistical terms, while the program began as a pilot with six schools in January 
1991, it encompassed 114 schools by the end of that same year. By 2004, the program had 7,831 
teachers and 378,208 students (Cuellar-Marchelli, 2003). In 2010, approximately 55% of rural public 
schools—which make up two-thirds of all schools in El Salvador—operated under the EDUCO 
program (Gillies, Crouch, & Florez, 2010). The continued popularity and expansion of the program 
was due, among other reasons, to its perceived success and to the fact that the ARENA party—with 
its preference for neoliberal reform—retained the presidency until 2009. 9 


The EDUCO Studies: A Critical Analysis 

It should be clear that each of the involved actors discussed above had a vested interest in the 
success of the EDUCO program. The Minister of Education had bet her career on community-level 
decentralization and had made it the primary strategy by which she sought to reform the governance 
of the education system. For their part, the representatives of World Bank who were working on this 
program saw it as a major opportunity both to make a name for themselves and to develop a policy 
that could be useful to their organization more broadly, that is, a policy that could be sold as a best 
practice to other countries. 1 " 

Yet, despite the excitement around this program during the early 1990s as it was being scaled 
up, the MINED and the representatives of the World Bank knew that they would need solid 
evidence which demonstrated that the program produced beneficial outcomes in order (a) to 
continue to promote the program as a central policy for education reform in El Salvador and (b) to 
be able to credibly promote the program internationally as a best practice (Edwards, 2013b). It was 
in this context of excitement and determination that the World Bank began to carry out evaluations 
of EDUCO. 

With the above in mind, the remainder of this section aims to critically evaluate six key 
studies that were carried out between 1994 and 2005 by the World Bank on the EDUCO program. 
These six studies were selected for critical review because they represent all the studies that the 
World Bank produced as impact evaluations. In other words, in accordance with the methodology 
of this article, and given that no other entity or actor generated any impact evaluations on EDUCO, 
these six studies represent the entire body of “legitimate” and “policy-relevant” knowledge that was 
created in order to evaluate whether the program worked in causal terms. 11 While these studies were 


9 See Edwards (2013b, pp. 54-55) for an update on the EDUCO program in the post-2009 time period. 

10 On this point, it is necessary to recall the tremendous popularity of decentralization as a theme within the 
international development industry during the 1990s (Edwards & DeMatthews, 2014) as well the 
predominance of the World Bank during this time vis-a-vis other multilateral organizations, especially in the 
education sector (Mundy, 1998). 

11 Going forward, it should be noted that there are additional EDUCO studies which were not included in the 
present review. These additional studies do not fit neatly into a pro-EDUCO trajectory and, whether due to 
their findings, methodology, authorship, timing or other reasons, they have not received the same kind of 
attention, nor have they wielded the same kind of influence. In contrast to the highly-cited and impactful 
studies critically re-assessed here, these other studies have employed methods which appear to be less 
rigorous and/or have offered findings which were less sanguine in nature. These studies include: Lindo 
Fuentes (1998) which presents findings from a qualitative case study of EDUCO funded by the World Bank; 
Sastry, Melamid and Ross (1995), which contains a cost analysis of the EDUCO program, also funded by the 



El Erograma EDUCO 


13 


integral to the continued success of EDUCO—within and outside El Salvador— during the latter 
1990s and 2000s, neither a comprehensive nor a critical analysis has been carried out of their 
methods and findings. In carrying out such an analysis, the goal, on one hand, to provide new 
insights into EDUCO’s knowledge base; that is, the purpose is to further inform what is known 
about the program. This is useful insofar as EDUCO in particular and community-level 
decentralization in general continue to be strategies that are considered for implementation by 
development specialists and policymakers around the world. On the other hand, through the analysis 
below, the goal is also to engage in a larger discussion of the role, implications and limitations of 
impact evaluations as a key element in the political economy of education reform. 

In what follows, the results of the critical analysis are presented. First, the context, methods 
and findings of each study are presented. Then, where appropriate, the limitations of each study are 
detailed. The subsequent section of this paper will reconsider what can reasonably be asserted with 
regard to the effects and lessons of EDUCO. The discussion below is summarized in Appendix 1. 

1. World Bank. (1994). El Salvador: Community education strategy: Decentralized school 
management. Washington, D.C.: The World Bank. 

Just before the end of 1994, the World Bank produced a report that contained the results of 
a small evaluation of the effects of the EDUCO program. Despite the limitations of the study, the 
results received significant attention, given that they were the first quantitative data available which 
could claim to approximate the program’s impact (World Bank, 1994). With a sample of 78 
communities, the World Bank applied an ex-post comparison 12 between two groups (EDUCO vs. 
traditional rural public schools) as well as ordinary least squares (OLS) regression, all using data 
collected in 1993. According to this report, and with reference to the two-group comparison, the 
evaluation showed better math test results for students in grades one and two, but it is important to 
note that no statistical controls were applied. Background characteristics (and only very few) are 
taken into account in just one instance—related to preschool academic achievement, and in this case 
the EDUCO variable is not significant. What was significant, rather, was family income (with there 
being a positive relationship between this variable and student achievement). While the larger point 
here is that the study lacked rigor, this did not prevent the Minister of Education and the World 
Bank from pointing to these results as evidence of EDUCO’s success. Not surprisingly, the 
appearance and reliance on these results coincided with the timing of a major political decision by 
the Minister of Education to continue and to further expand the EDUCO program through 
additional World Bank loans. Later years would see the production of five additional (and more 
rigorous) studies, and these would constitute the international evidence used to promote the 
program, as discussed below. 


World Bank; Parandekar (2002), a quantitative study by a World Bank researcher that looks for lessons 
generally from EDUCO related to the decentralization of education; Reimers (1997), an independent study of 
the characteristics and implementation of the EDUCO program; and Cuellar-Marchelli (2003), another 
independent study that assessed the overall experience of EDUCO along various dimensions.Each of which 
has been carefully reviewed in Edwards (2013b). Other, more recent reports and studies that document the 
operation of the EDUCO program are: Avila de Parada & Landaverde de Romero (2007); Ayala (2005); 
Briones et al. (2006); Gillies, Crouch & Florez (2010); SIMEDUCO (2011); Srygley (2013); and World Bank 
(2009). All of studies cited here are available from the authors. Due to the present focus on impact 
evaluations, these other studies were not included here, though future research should further explore their 
insights. 

12 Meaning that the EDUCO schools and the traditional schools were selected for comparison after the 
EDUCO intervention began. 
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2. Umanzor et al. (1997). El Salvador’s EDUCO program: A first report on parents’ 
participation in school-based management. Working Paper Series on Impact Evaluation of 
Education Reforms, No. 4. World Bank. 

The second study was conducted by the World Bank and a few staff from the MINED. It 
was an impact evaluation and it relied on data collected in 1996 (most future studies would return to 
this data as well) through a random sample of 311 schools. The population of schools from which 
the sample was drawn had to have a third grade section with at least 10 students. At each school, in 
addition to administering language and math tests to third-grade pupils, the team interviewed and 
gave a questionnaire to the director (or person in charge), the third-grade teacher, five randomly 
selected fifth-grade students, their parents, and two members from the ACE. The questionnaire 
collected background data. Although the sample initially included four types of schools - i.e., 
EDUCO, mixed, 1 ’ traditional public, and private - the 1997 impact study only looked at EDUCO 
schools and traditional public schools located in rural areas. 

Underlying this data is an important feature to note before continuing with the study’s 
results. Though the EDUCO sample was identified in a random way, this is negated by the fact that 
the MINED initially chose the communities which would participate in a non-random way. That is, 
as discussed previously, the MINED purposefully selected the poorest and most disadvantaged 
communities for the program (economists refer to this strategy as endogenous program selection), 
thus indicating that selection bias would be a problem inherent to the quasi-experimental studies 
conducted later. 

With this data, Umanzor et al. (1997) ran a series of ex-post matched comparisons between 
the EDUCO schools and the traditional public schools on a number of characteristics of interest. 
They focused on associations between school and family characteristics and the outcomes observed 
in the school. They found both similarities and differences in general characteristics across groups. 
For example, on average, and to a statistically significant degree, EDUCO teachers had a higher level 
of education, but less experience. This is because, at the outset, ACEs were required to hire teachers 
who had at least a university level teaching certificate. The excess supply of university-trained 
teachers in the early 1990s also contributed to this. EDUCO schools also had less access to 
electricity, but more of them had classroom libraries (most likely because a separate USAID 
program funded classroom supplies). Despite better libraries, EDUCO schools did not have more 
textbooks. Additionally, EDUCO students were found to live in much more rudimentary homes (as 
in, without electricity or piped water, but with dirt floors), yet parents were not found to help with 
homework any less than parents in the comparison group. In terms of school-community 
interaction, ACE members felt that they had more influence in school decisions, though they did 
not actually make more decisions in the school’s management (Umanzor et al., 1997). ACE 
members also visited their community’s school more often than did parents on the school councils 
of traditional rural schools. Likewise, EDUCO teachers met with parents more often and contacted 
parents more frequently when students were absent. Lastly, with regard to the variables that have 


13 LTntil 1999, the MINED allowed EDUCO and traditional schools to co-exist through what it called ‘mixed’ 
schools (Lindo Fuentes, 1998). Here, a traditional school would also receive funding from the EDUCO 
program and the community would establish an ACE. With the funding received, the community would hire 
a teacher for lower grades (grades 1-3). The school would then use its non-EDUCO budget to pay a teacher 
to provide grades 4-6. As mentioned, the EDLTCO program was initially targeted to communities where 
educational services were not provided during the civil war. However, as the MINED began to scale up the 
program it did so by creating ‘mixed’ schools, as well. 
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received the most attention, there were differences in neither student absenteeism, dropout, grade repetition nor 
student achievement on either the math or language tests (Umanzor et al., 1997). 

Some of these differences are cause for optimism — as with ACE parents being more 
involved in the school and EDUCO teachers meeting more with parents. Nevertheless, educational 
interventions are assessed primarily in terms of their ability to increase student achievement. 
Moreover, in this case, the theory of the EDUCO program is in part based on the notion that 
community-level management would result in increased teacher effort, which would lead to better 
student learning. Although the comparison of the treatment and control group in this study did not 
show that EDUCO students had significantly better achievement than students in mral traditional 
public schools, these students also did not score significantly worse than their counterparts in public 
schools. Umanzor et al. (1997) took this as a positive sign, given the relatively worse economic 
conditions of EDUCO students, and thus conclude that EDUCO students thus exhibit “better than 
expected academic performance” (p. 26). To be sure, and as will be discussed, the conclusions by 
Umanzor et al. (1997) are the most justifiable of the studies reviewed here. That said, it may be that 
the positive results observed were not the result of the EDUCO program, per se, but rather a result 
of the pre-existing social capital that had accumulated during the civil war in the EDUCO 
communities (many of which were FMLN communities, at that time), when they were forced by 
their circumstances to work together to provide basic services, such as education (ADES, 2005; 
Aguilar, Becerra, Burgos, Castro ,& Garcia, 1993; Alvear Galindo, 2002; Cmz, 1994, 2004; MINED, 
1990, 1994). 

3. Jimenez, E., & Sawada, Y. (1999). Do community-managed schools work? An evaluation 
of El Salvador’s EDUCO program. The orld Bank Economic Review, 13 (3), 415-441. 

The third study - by Jimenez and Sawada (1999) - emerged two years later. 14 Here, Jimenez 
and Sawada (1999) employed more advanced econometric techniques to assess whether EDUCO 
had an effect on either student achievement or student absences. In order to answer these questions, 
they used the same 1996 data as Umanzor et al. (1997) and applied regression analyses along with 
the Heckman two-step correction (Heckman, 1979) in order to attempt to correct for the sample 
selection bias inherent in the EDUCO treatment group. 15 In their study, they successively added 


14 It should be noted that Emmanuel Jimenez was at that time a member of the World Bank’s influential 
Development Economics Vice-Presidency, and was well respected among his peers. The fact that he took up 
the cause of EDUCO by doing research on it attracted the attention of others within the Bank. 

15 The Heckman two-step correction is employed where there are issues with sample selection bias. Sample 
selection bias is normally a problem for regression analysis because, when the treatment and comparison 
groups systematically differ on a range of characteristics, we cannot be sure that the effect observed for the 
treatment (e.g., EDUCO) is unbiased. That is, the effect (or lack of it) could be due to unobserved variables 
related to the treatment (labeled by economists as endogeneity). Under endogeneity, we can also not be sure 
of the extent to which the estimates for the other variables in equation are biased as well (Berk, 1983). The 
best way to address this issue is to first draw a representative random sample of participants from the 
population of interest. In most cases this is not possible, especially in social science research. Such was the 
case with EDUCO, where participating communities were not chosen randomly and where participating 
communities differed systematically from a nationally representative sample. 

Thus, to deal with this common issue, James Heckman developed his now famous correction 
technique which treats sample selection bias as specification error (Heckman, 1979). Per this technique, a 
separate estimator term is added to the principal regression equation of interest (i.e., the “substantive 
equation”). This term is arrived at by estimating through a (probit) selection equation the likelihood of a 
student, for example, being placed in an EDUCO school, given a set of selected characteristics. This 
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controls for student, family, teacher, classroom, school, and community participation characteristics. 
Throughout, the comparison group is traditional mral schools. 

In their discussion of student achievement, the authors state: “EDUCO’s ... effect on 
language test scores is positive and significant, while its effect on math performance is positive and 
not significant” (Jimenez & Sawada, 1999, p. 428). Yet, this statement is quickly followed by a 
concession: “Our measure of EDUCO’s advantage in language may be imprecise. The estimate of 
the EDUCO coefficient is sensitive to the specification of the participation equation” (p. 428). What 
they allude to is that their more completely specified models clearly show that the results for both language and 
math achievement are not significant. Nevertheless, the authors make inferences in the concluding section 
to their paper from models missing controls for school inputs or community participation. They 
state: “[EDUCO] has improved language scores” (p. 440). 

In addition to the specification issue, there is the matter of endogeneity stemming from 
sample selection bias. That is, the results provided by Jimenez and Sawada (1999) are confounded by 
the fact that the students who attended EDUCO schools also possessed unobserved characteristics 
that correlated with student achievement outcomes. Jimenez and Sawada (1999) recognize this 
when they write, “The error terms of the participation and achievement equations are negatively 
correlated. This means that EDUCO students have unobserved characteristics that are negatively 
correlated with achievement test scores” (p. 428). Under conditions of endogeneity, one cannot truly 
parse out the impact of the intervention on student achievement because the participation of the 
student in the intervention is correlated with other factors which are also correlated with student 
achievement. 

Additionally, alternative conclusions can be drawn from the student achievement regression 
results, keeping in mind their limitations. In math, for example, although the EDUCO variable is not 
significant, what does contribute to improved test scores is the availability of sanitary services. With 
regard to language test scores, it is having a classroom library that contributes positively. For both 
math and language, more frequent visits to the school by ACE parents is positively associated with 


probability, then, is fed back into the substantive equation as part of a ratio (the inverse Mills ratio) that 
represents the probability that a child with given characteristics will be excluded from the sample from which 
results are generalized, conditional on participation in EDUCO (Berk, 1983). Ideally, then, the added 
estimator is a sample selection correction term that adjusts for the fact that certain characteristics are 
overrepresented in the sample being examined. From the regression output, one can then interpret a 
statistically significant result for this term’s coefficient as indicating that there is sample selection bias. That 
is, a statistically significant coefficient signals that certain characteristics are over or under-represented. 

Fundamental issues surface in practice, however. For a thorough review of the many issues that can 
and do arise in practice through the use of the Heckman two-step correction, see Bushway, Johnson and 
Slocum (2007). For example, if the same variables appear in both the selection equation and the substantive 
equation, the issue of multicollinearity arises (Bushway, Johnson & Slocum 2007). That is, in concrete terms, 
if the same variables used to predict a student’s participation in EDUCO are also used to predict a student’s 
test scores, then the correction term added to the equation for test scores may well co-vary with the other 
independent variables. What’s more is that this technique assumes that the error terms for both the selection 
and substantive equation are jointly normal, meaning that they are independent of each other (Heckman, 
1979). However, the error terms will be correlated if there are unobserved characteristics (omitted variables) 
relevant to the dependent variables of both the selection and substantive equations. It should be noted that, 
in the case of EDUCO, this is almost certainly the case. The reason is as follows: Socioeconomic conditions 
determined who participated in EDUCO; these same poverty related characteristics always influence student 
achievement. Ultimately, then, although this fix works well in theory, in practice it actually worsens issues of 
misspecification because there are now two linked regression equations to specify correctly, and which 
themselves must not be correlated. 
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higher student achievement. The data thus support the conclusions that schools in which parents are 
involved and which possess basic resources encourage better student performance. Given that the 
focus of the evaluation was to determine EDUCO’s impact, these findings are not highlighted. 

The second question posed by Jimenez and Sawada (1999) is whether there is an EDUCO 
effect on student attendance. They use the same data and controls as with their models for student 
achievement, changing only the dependent variable to indicate the number of days the student was 
absent in the month prior to the survey. Elere, they find that EDUCO significantly reduces the 
number of days a student was absent. They thus conclude that the accountability relationships 
instituted between parents and teachers which EDUCO are responsible for reducing student 
absences. 

However, if we decompose the effect of the EDUCO variable according to when the 
EDUCO schools were built, the picture changes (Jimenez & Sawada, 1999). If the EDUCO school 
was built between 1991-1995, there was no effect; however, if the school was built in 1996, it 
significantly decreased student absences. As opposed to attributing the effect to relationships of 
accountability, the result could simply be due to the fact that students received a new school in the 
same year that the survey was administered. The presence of a new school - where there had not 
been one previously - could significantly increase student attendance. 

Nevertheless, the authors chose to highlight a particular conclusion, one that resonated with 
the theory of community-level accountability relationships and their ability to lead students and 
teachers to perform better (Jimenez & Sawada, 1999). These conclusions are powerful, particularly 
given the fact that they are offered by the World Bank and the fact that they were derived from 
methods that are widely seen as rigorous and objective. To that end, the literature shows that other 
authors have repeated these conclusions, as did Barrera-Osorio, Fasih, Patrinos, and Santibanez 
(2009) in their review of the effectiveness of school-based management in developing countries. In 
this way, over time, conjecture can contribute to consensus around the idea that community-level 
decentralization can lead to fewer student absences. 

4. Sawada, Y. (2000). Community participation, teacher effort, and educational outcome: 

The case of El Salvador’s EDUCO program. Michigan, USA: The William Davidson 
Institute, University of Michigan Business School. 

The fourth study was conducted by Sawada (2000) and attempted to determine if the 
EDUCO program had an effect on student achievement, teacher pay, teacher effort, or classroom 
inputs. Numerous models were estimated using largely the same data and controls as were employed 
by Jimenez and Sawada (1999), the difference being that Sawada (2000) uses instmmental variables 
in an attempt to remedy the endogeneity which results from the nature of the sample. 16 In the end, 
four conclusions were drawn, each of which is reviewed here. 

The first conclusion stated that community participation was positively related to teacher wages. However, 
although Sawada (2000) mns eight different models to test this, under the most complete 
specification neither the community participation variable nor the EDUCO variable is significant. 


16 Instrumental variables are employed in regression analysis under conditions of endogeneity. The idea is 
that one seeks to “find a variable (or instrument) that is highly correlated with program placement or 
participation but that is not correlated with unobserved characteristics affecting outcomes” (Ishandker, 
Koolwal, & Samad, 2010, p. 87). Instrumental variables must be selected thoughtfully, as a weak instrumental 
variable - one that is correlated with unobserved characteristics or omitted variables — can worsen the bias of 
coefficient estimates. Ideally, the instrumental variable serves as a proxy for program participation while 
eliminating issues of endogeneity. See the previous footnote for more on endogeneity. 
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The author refers in his conclusion to an OLS regression which did not control for teacher 
characteristics. Moreover, in this particular model, while it is true that community participation 
(defined as the number of times parents on the ACE visit the school monthly) is positively 
associated with increased wages, what Sawada (2000) chooses not to mention is that EDUCO is 
significantly and negatively related to wages. Thus, while Sawada (2000) finds a significant effect of 
community participation, this is not related to the EDUCO program. In contrast, the EDUCO- 
related finding - that EDUCO teachers receive less compensation than their traditional school 
counterparts — is not highlighted. 

Second, Sawada (2000) highlights the relationship between teacher effort and community participation in 

EDUCO schools. Here, the model does indeed show a statistically significant relationship between 
school visits by ACE members and the number of hours teachers spend each month meeting with 
parents. However, the latent constructs, as measured and specified here, experience simultaneity 
and, as such, it is not possible to separate the effect of community participation from teacher effort. 
This is not to say that there is not a relationship between the two variables. To be sure, parents 
spending more time at school would be intertwined with the number of hours teachers spend 
meeting with parents. What remains unclear is the nature and directionality of the relationship. 
Nevertheless, Sawada (2000) concludes: “Community participation seems to enhance the teacher 
effort level” (p. 26). 

For the third finding, Sawada (2000) reports that there is a connection between community participation and 
teacher-student ratio. In particular, the finding is that the former causally decreases the latter. This is a 
curious findings because the relationship between community participation and teacher-student ratio 
is not clear. That is, one wonders why teacher-student ratio would be a function of community 
participation. While Sawada (2000) does not offer an explanation of why this would be, it is 
suggested here that this may be the result of the nature of the sample: EDUCO schools were located 
in particularly rural areas, where there tended to be fewer children per teacher. 

Fourth andfinally, Sawada (2000) speaks to student achievement. Although the EDUCO variable 
was not in any of the model specifications significandy related to test scores, the study is bookended 
by statements which, at first glance, indicate the contrary. At the beginning and at the very end of 
the paper, Sawada (2000) writes, “We observe consistendy positive and significant EDUCO 
participation effects on standardized test scores” (pgs. 1, 25, emphasis added). What he refers to, 
however, is not a variable that represents parental participation in EDUCO, but rather a variable for 
parental participation generally. The problem with this wording is that the parental participation 
variable which was shown to be associated with student achievement is not specific to EDUCO. 
Instead, it is a variable that represents participation in schools generally — by either parents of 
EDUCO students or parents of students in traditional mral public schools. 1 Such statements can 
filter through to future reviews of empirical work on EDUCO and decentralization more generally. 

5. Jimenez, E., & Sawada, Y. (2003). Does community management help keep kids in 
schools? Evidence using panel data from El Salvador's EDUCO program. Discussion 
Paper, Center for International Research on the Japanese Economy. Tokyo, Japan: 
University of Tokyo. 

In the fifth study, Jimenez and Sawada (2003) attempt once more to uncover an EDUCO 
effect. This time, they return to the question of whether participating in EDUCO is associated with 
staying in school over the two-year period 1996-1998. To answer this question new data collected in 


17 The parents associations for traditional schools were known as Sociedades de Padres de Familia (SdPF), or 
Family Parent Societies (Sawada, 2000, p. 3). 
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1998 (from student learning tests and teacher and parent surveys [Lindo, 1998, p. 21]) is added to 
the 1996 data and then run through regression analyses (specifically, they use probit models) which 
utilize the Heckman two-step correction for sample selection bias and the same controls as their 
other studies (Jimenez & Sawada, 1999; Sawada, 2000). 

Once again, the authors acknowledge the inherent limitations they face due to sample 
selection bias and the effects of this on their results. They reveal that the error terms for the 
selection and substantive equations are negatively correlated. As a result, Jimenez and Sawada (2003) 
note that “unobserved characteristics regarding the children, households, and communities might 
positively affect the likelihood of selecting an EDUCO school and might negatively ... affect a 
student’s decision to continue in school” (p.21-22). The implication is that their findings may suffer 
from endogeneity and, as such, may not be valid. Nevertheless, the findings of Jimenez and Sawada 
(2003) can be found cited in the literature on education decentralization in developing countries. 

One example is Barrera-Osorio, Fasih, Patrinos and Santibanez (2009), who cite results from model 
1 in the study by Jimenez and Sawada (2003). While the results cited indicate that attending an 
EDUCO school increases the probability of remaining in school for two years by 64%, we cannot 
really be sure due the issue of endogeneity. 

6. Sawada, Y., & Ragatz, A. (2005). Decentralization of education, teacher behavior, and 
outcomes: The case of El Salvador’s EDUCO program. Incentive to improve teaching: 
Lessons from Latin America, (pp. 255-306). Washington, D.C.: World Bank. 

The sixdi and final study came about in 2005, this time by Sawada and Ragatz (2005). It 
builds on Jimenez and Sawada (1999). In this incarnation, the study asks how decentralization affects 
administrative processes, teacher behavior, and student achievement. In addition to using the 
Heckman two-step correction procedure, it also makes use of propensity score matching - a strategy 
through which the compared treatment and control communities are matched along a series of 
characteristics to ensure that they are as identical as possible, given the communities in the sample. 

In their analysis, Sawada and Ragatz (2005) eliminated the mixed schools included by Jimenez and 
Sawada (2003), thus reducing the sample to 37 EDUCO schools and 96 traditional schools. In some 
models, the same controls as before were used. Those instances where the controls were different 
are discussed below. 

The results are interesting in that they are not what the authors expected. In terms of 
administrative processes, the only activity to occur more often at the school level in EDUCO 
schools concerns hiring and firing teachers (Sawada & Ragatz, 2005). Neither determining teacher 
salaries, giving teachers incentives, evaluating teachers, nor spending school money occurred more 
often at the school level in EDUCO than in traditional public schools. Not surprisingly, the results 
show that teachers’ association relations occurred more often in traditional mral public schools. This 
was the case because teachers hired by ACEs are non-union, and thus there was no teachers’ 
association activity to observe. 

For teacher behavior, interpreting the results is like reading an Ouija board. There are 
models for 19 different dependent variables. Each dependent variable tells us something different 
about teacher behavior - for example, how many hours per week the instructor teaches, how many 
hours s/he meets with parents, or how often s/he is absent. Further complicating the picture is that 
each model is then run four different ways - using (a) standard (that is. Ordinary Least Squares) 
regression with 42 independent variables, (b) the Heckman two-step procedure, (c) revised OLS 
regression with 32 independent variables, and (d) propensity score matching with 32 independent 
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variables. 18 The problem across these 76 different combinations is that there is tremendous 
variability in terms of what is significant. Those results that are significant under propensity score 
matching with 32 controls are then nonsignificant in the OLS regression with 42 controls, for 
example. Elsewhere, the Heckman models return significant coefficients, but, in these cases, none of 
the other models do. In the end, there is not one outcome variable that is significant across all four 
forms of regression. Sawada and Ragatz (2005) still report, however, that “many key measures 
survived the bias tests, including meeting with the parents, teacher absence, and hours dedicated to 
teaching” (p. 295). 

In addition to the above finding, the authors offer two additional conclusions based on the 
regression results. The first is as follows: 

The results indicate that community participation seems to enhance the teacher 
effort level, possibly because of intensive monitoring of teacher behaviors and the 
implicit threat that exists because hiring and firing of teachers occurs at the 
community level (Sawada & Ragatz, 2005, p. 295). 

And, second: “Community participation not only uses relevant information that outside government 
agencies are not likely to have, but also imposes commitment on teachers, which leads them to exert 
greater effort” (p. 295). 

It is suggested here, however, that these assertions are not borne out by the regression 
results, given the variability and inconsistency across models discussed above. It seems that the 
authors were either confident in the regression results or had a predilection to explain the results in 
terms of accountability relationships that resonate with the World Bank’s theory of community-level 
education management decentralization, or both. In the end, and in contrast with the interpretation 
by Sawada and Ragatz (2005), it is suggested here that the true effect is not known for any of the 
significant variables presented by the authors. As Klees and Edwards (2014) assert: 

Regression analysis theory ... offers no guidance as to how inaccurate the resulting 
regression coefficients are under real-world misspecification conditions... . With so 
many alternative specifications from which to choose, regression analysts can almost 
always find their favorite variable to be significant, and thus the empirical literature 
becomes a debate over who has the best specification (p. 34). 

The final question of this study pertained to student achievement. This time, the results look 
promising, for language scores. We are presented with the output for a handful of coefficients across 
four specifications. Three of the four models show significant effects of being in an EDUCO school 
compared with being in a traditional rural public school. What the models do not show, however, 
are the authors’ controls, though the footnote to the table does note that child and household 
characteristics were taken into account. Yet what the authors choose not to mention is that many of 
the available controls were not applied. That is, Sawada (2000) and Jimenez and Sawada (1999) ran 
the same models with additional controls for teacher quality, classroom quality and school quality, 
and under those circumstances they did not find an EDUCO effect on student achievement. One 
wonders if Sawada and Ragatz (2005) purposefully specified their models to generate significant 
effects. There is no way to know for certain. Nevertheless, the fact that they chose not to use the 
available controls puts in question the conclusion they offer: That “EDUCO program governance 


18 Propensity score matching could not be run using all 42 variables because the communities were too 
dissimilar, according to Sawada and Ragatz (2005). 
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leads to better effort of teachers, which improves educational outcome” (Sawada & Ragatz, 2005, p. 
299). 19 

Many of the interpretations offered by Sawada and Ragatz (2005) raise concerns. 
Nevertheless, one conclusion suggested by the data is that ACE parents, compared with parents on 
the school councils of traditional rural public schools, have and use the ability hire and fire teachers 
more. Given the provisions on which the EDUCO program rested all along, this is to be expected. 
Other findings from this study are more controversial and uncertain, and seem to be intended to 
positively skew the perception of EDUCO. 

Discussion: What Do We Actually Know About EDUCO? 


The previous section clearly calls into question the findings that have been asserted in the 
most widely-known and widely-cited studies of EDUCO. In light of the critical review presented 
above, these studies can interpreted as repeated attempts to find evidence that supports the 
beneficial effects of a particular version of decentralization. It was shown that World Bank 
researchers and publications, in many instances, offered interpretations which were questionable 
(based on the limitations of their methods and data and the possible alternative explanations). In 
view of the reappraisal above, which is summarized in Appendix 1 to this paper, it is argued here 
that not one of the studies reviewed definitively shows that the EDUCO program produced positive 
and significant results on the variables of most interest to the World Bank and the MINED. To the 
contrary, it is concluded that the EDUCO program does not lead to improved student achievement, 
student attendance, student retention, or teacher absenteeism when the available controls are 
applied. 

The one finding which stands out across the studies is that EDUCO schools did no worse 
(in statistically significant terms) on student achievement than traditional rural public schools. 
EDUCO proponents have consistently interpreted this as a positive finding: since EDUCO schools 
were located in the most disadvantaged areas of the country, one might have expected them to 
perform much worse than their public school counterparts. However, given that EDUCO schools 
performed just as well (i.e., no worse), EDUCO advocates conclude that, under the EDUCO model, 
we can introduce an arrangement that is more accountable without sacrificing quality. 

Yet one cannot be sure that the effect is due to the EDUCO model. First, it must be 
remembered that the data for these studies were collected during 1993-1998; second, it must be 
remembered that EDUCO schools were, at the outset, purposefully targeted to FMLN communities 
(Reimers, 1997) as a way to undermine the system of popular education that had developed during 
the civil war (Edwards, 2013b, p. 224). It should be noted that the popular education system was not 
only present in the areas controlled by the FMLN but it was also substantial in its reach: by the early 
1990s, this system included approximately 1,000 teachers and 13,500 students (Alvear Galindo, 

2002, pps. 189, 200), although according to Reimers (1997) this system served as many as 17,000 
students. The implication is that, during the time when data was being collected for the reviewed 
studies, the communities included in the sample were in fact FMLN communities that had previous 
experience with self-organization through the system of popular education as well as through 
cooperative and democratic forms of agricultural production, communication, housing, and 
community government (ADES, 2005). This realization is problematic for the conclusions drawn by 


19 When asked in person about the lack of use of controls in Sawada and Ragatz (2005), professor Sawada 
explained that fewer controls were used because the available “dataset was not suitable for regression 
analysis” (personal communication, October 3, 2014). 
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EDUCO supporters because it means that the findings which they have attributed to the EDUCO 
model could rather be the result of the practice with self-organization that these communities had 
during the civil war as part of their efforts to prepare for the reality of a socialist and cooperative 
society after the war, assuming the FMLN won (Montgomery, 1995). Put differently, as opposed to 
attributing the results to EDUCO the results could just as likely be attributed to the pre-existing 
social capital possessed by the communities which participated in EDUCO. 

Given the plausibility of this alternative explanation, there are now no results which suggest 
the superiority—or even the equivalence—of EDUCO to traditional mral public schools, 
particularly in terms of such outcomes as student achievement, student attendance, student 
retention, or teacher absenteeism. With regard to parental participation, Umanzor et al. (1997) did 
show that parents in EDUCO schools visit their child’s classroom more times per month (5.7 times 
vs. 1.6 times) and spend more time meeting with teachers (EDUCO teachers spent 4.9 hours/week 
meeting with parents, versus 2.9 hours/week for non-EDUCO teachers). But again, these findings 
may be spurious because the groups of parents being compared were likely not the same in terms or 
their social capital, as explained above. The results may have been equivalent for EDUCO and 
traditional rural schools if the parents from the latter group had the same pre-existing social capital. 

Interestingly, however, in terms of lessons learned, our reappraisal of the evidence actually 
suggests a contrary conclusion to that offered by the authors of the studies. To reiterate, if we recall 
that the “EDUCO communities” in the impact evaluations are FMLN communities that had 
adopted the EDUCO program, then any positive benefits observed may be a reflection not of the 
design of the EDUCO program but rather a reflection of the experience of cooperative self¬ 
organization and the social capital that consequently accumulated with the parents who, under 
EDUCO, served on the school councils. Seen in this light, the impact evaluations of EDUCO are, 
ironically, evidence in favor of the opposition, that is, evidence in favor of those same communities 
that the government sought to undermine, and which the government did in fact undermine 
through the requirement that EDUCO schools could only hire certified teachers, which necessarily 
prevented the ACEs from hiring popular education teachers, since they had low levels of education 
without the credentials specified under EDUCO’s regulations. But the fact that EDUCO 
successfully subjugated FMLN communities to the official system—despite proposals from popular 
educators to allow for two parallel systems (Aguilar, Becera, Burgos, Castro, & Garcia, 1993)—does 
not erase the social capital that accrued to parents nor their prior experiences with self-organization, 
both of which informed and arguably improved their work under EDUCO when it came to hiring 
and supervising teachers and managing the school budget among other tasks. Thus, the findings by 
Umanzor et al. (1997) that EDUCO parents visited their children’s classrooms more often and spent 
more time meeting with teachers is likely the result of efforts at community organization that go 
back to the 1970s, when mral Salvadorans began to mobilize in response to governmental repression 
and in accordance with the critical teachings of liberation theology (Montgomery, 1995). These same 
communities later became affiliated with the FMLN and put into practice critical teachings through 
their self-managed community schools and the methods of critical educators such as Paulo Friere. 
Ultimately, then, this critical review suggests that the experience of EDUCO supports forms of 
community education management that are rooted in broad, community-wide cooperative 
organization, solidarity and popular education, as opposed to a more narrow, neoliberal version 
based on accountability, student test scores, the ability to fire teachers, and abstract notions of 
efficiency. These two versions of community participation in education governance have been 
referred to as progressive and neoliberal, respectively (Edwards & Klees, 2015). 

In sum, then, the same studies which have supposedly shown the benefits of the EDUCO 
program can be interpreted differently to offer alternative conclusions. These alternative conclusions 
clearly cast doubt on the advisability of EDUCO as a public policy. Furthermore, by placing our 
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critical review in the political economic context of the 1990s, it is suggested that the alternative 
interpretations offered here indicate that EDUCO, on the whole, was: (a) a program which did not 
improve key indicators of quality, (b) imposed costs on the mral parents (who volunteered their time 
on the ACEs and who helped to build and repair the schools in their communities), (c) sought to 
weaken the teachers unions (by instituting one-year contracts for EDUCO teachers), and (d) was 
directed at incorporating FMLN communities into the official education system (by replacing their 
popular education teachers and schools with the EDUCO program and with teachers from outside 
their communities). Of course, this alternative interpretation highlights the political motivations 
behind EDUCO. But it needs to be remembered that EDUCO was always a political decision. 
Moreover, and not surprisingly, it has been in the realm of politics that the EDUCO program has 
continued to have relevance, thanks in no small part to the studies reviewed here, as is discussed in 
the next section. 


National and International Implications 


Within the particular political-economic context of the 1990s and 2000s, these studies have 
contributed to numerous developments, both inside and outside El Salvador. In what follows, the 
implications are discussed in national and then international terms. Keep in mind that the 
implications discussed below are not presented as being the result of the EDUCO studies alone; 
rather, it is suggested that the EDUCO studies are one important factor among many within the 
historical context characterized earlier in this article. 

National Implications 

First, within El Salvador, these studies initially influenced the decision in 1994 to scale up the 
program countrywide, and then later served to maintain the focus of the MINED on EDUCO, even 
after the transition to a new political administration in 1998 and again in 2003. Gillies (2010) has also 
noted this latter point. Of course, the other side of the coin is that these studies, and the 
justifications they offered, constituted significant obstacles to overcome for those who opposed the 
program, particularly because of their econometric nature, and thus the perception that their 
findings were definitive, not to mention the difficult-to-unpack nature of the methodology itself. At 
once, then, these studies provided political capital to those in power while also raising barriers to 
reform for those who sought to modify or repeal the program. 

A second (and ironic) implication is that these studies—which popularized EDUCO for 
enhancing teacher accountability, for reflecting a neoliberal logic with regard to efficiency and 
effectiveness—actually helped to extend a model with roots in the FMLN’s approach to 
development, which was based in solidarity, community empowerment, and popular education 
(ADES, 2005; Edwards & Avalos, 2015). However, as noted, the FMLN model was sanitized and 
reframed according to market-based principles. Amid the focus on this new management model, 
what received less attention was the fact that, through EDUCO, a second-class system of schools 
was created that continued to marginalize large portions of the student population. This outcome 
resulted from the fact that, in addition to hiring less-experienced teachers, EDUCO communities, 
which already tended to be located in poor mral areas, were also expected to voluntarily contribute 
to the construction, maintenance, and management of their community school (note that none of 
the parents on the ACEs received compensation for their time). 

More generally, and perhaps more perniciously, these studies contributed to the 
development and general acceptance of myths around the EDUCO program. In El Salvador, during 
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the last 20 years, it has become common to espouse the notion that EDUCO is a program which 
introduces teacher accountability and which produces better student achievement, among other 
things. By repeating these assertions, researchers, political actors and education commentators have 
helped to spread a kind of “common sense” around EDUCO. The present article is an attempt to 
provide an alternative narrative to the “common sense” that heralds market-based forms of 
accountability. 

Third, in that EDUCO was the education reform superstar of the ARENA party, it stands to 
reason that this program—and its positive evaluations—contributed to this party’s continued 
success in national elections, even if only as one element among many. 

International Implications 

Internationally, the reviewed studies have been used to promote the implementation of a 
model that not only transfers to the community part of the responsibility for education governance, 
but also transfers part of the cost of educational provision. That is, while studies such as those on 
EDUCO have provided the justification for countries to experiment with community-level 
decentralization, in practice, these studies have served as political cover for the introduction of 
decentralization policies that require (whether officially or unofficially) that community members (a) 
make donations to the school, (b) help with building and repairing school buildings and other 
structures (such as latrines), (c) contribute to school management (e.g., through monitoring teachers 
and principals), and (d) participate in administrative or supplemental tasks (e.g., the development of 
school improvement plans or the provision of school snacks for students) (Edwards & DeMatthews, 
2014). 

The second international implication, which relates to the first, is that these studies have 
promoted the belief among development practitioners that EDUCO-style community 
decentralization is a preferred reform for education systems around the world. A prominent example 
is the World Bank. As the studies reviewed here began to be produced, they garnered attention 
outside of the staff members who initially worked on the EDUCO program, including prominent 
staff within this organization. Indeed, as Gershberg, Meade, and Andersson (2009) note, based on 
personal observation, “Jimenez and Sawada (1999) was influential in convincing World Bank staff 
that EDUCO was a successful model” (p. 198). 211 Moreover, the evidence base constituted by both 
Jimenez and Sawada (1999) and Sawada (2000) grabbed the attention of the authors of the World 
Development Keport 2004 (WDR 2004), which is the flagship publication of the World Bank (World 
Bank, 2003). In this Report, which has come be regarded as a guide for how to introduce relations 
of accountability in public sector reform in developing countries, the EDUCO program was 
highlighted as the key model to emulate in the education sector. By being included in the WDR 
2004, EDUCO’s reach among development professionals was thus amplified, thereby increasing the 
influence of this model on the global education reform agenda. Were it not for the findings offered 
in the EDUCO studies, this would not have been possible. For additional examples of the ways that 
EDUCO was invoked in the 1990s and 2000s, after these studies began to come out, see Appendix 
2 . 

Third, in raising the profile of the EDUCO program, the EDUCO studies also helped to 
raise the profile of the personnel from the MINED who were instrumental in institutionalizing and 
implementing the EDUCO program during the 1990s. For example, following Cecilia Gallardo de 
Cano, who was the Minister of Education (1990-1998) when EDUCO began, three of the four 
ministers of education during 1998-2009 (after which point the presidency passed from ARENA to 


20 Alec Gershberg served as a senior education economist for the World Bank during the late 1990s. 
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the FMLN) previously either worked for the EDUCO program or otherwise contributed to its 
furtherance. 21 Of course, without the positive evidence presented by the EDUCO studies reviewed 
here, these actors would have lacked the ability to claim success, or to have success attributed to 
them, thereby making it more difficult to generate the political capital necessary to assume such 
high-profile positions. This not to say that it would have been impossible for such actors to reach 
such prominent positions, only that it would have been less likely. 

Fourth, these studies helped to generate interest from other countries. Governmental study 
groups came from, at least, Argentina, Bangladesh, Brazil, Chile, Equatorial Guinea, Honduras, 
Niger, and Thailand, as well as additional study groups from Africa, the Caribbean, and Europe 
(Meza, Guzman & de Varela, 2004a). MINED representatives themselves also went to numerous 
other countries. MINED interviewees in Edwards (2013b) mentioned that, over time, they were sent 
by the World Bank to such countries as Bolivia, Brazil, China, Colombia, Guatemala, Honduras, 
Mexico, Spain, and the United States. Not surprisingly, the World Bank played a key role in 
financing these study trips, which was not an unusual practice during the 1990s. Separately, these 
impact evaluations—together with the movement of World Bank staff—have influenced the design 
and adaptation of very similar models at least in Afghanistan, Burkina Faso, Guatemala, Honduras, 
Nepal, Niger, Senegal y Uganda, although it should be noted that there may be more cases of the 
direct influence of the reviewed studies, only that it is not possible to identify them because the 
individuals inspired or unknowingly influenced by the evidence on EDUCO do not always cite it 
publically as a point of reference (Najjumba, Habyarimana, & Bunjo, 2013; Sawada et al., 2014; 
Shibuya, 2013). To that end, an education specialist from the World Bank with over 20 years of 
experience in El Salvador and other countries explained how the lessons from EDUCO have 
become part of “international knowledge,” such that development practitioners and governmental 
functionaries in other countries are unaware that they are following the example of El Salvador: 

Most development agencies now have learned about those models that happened, 
you know, more than 20 years ago, but, 20 years ago, actually, to go from something 
that was a natural process [in El Salvador, in the absence of governmental support] 
and then saying, ok, first we need to set up an organization to create a community 
body and let’s give some formal role to parents and then they need to plan and create 
a school improvement plan and they need to have committees where students come 
together with teachers and parents and they execute those plans, and we need a 
mechanism to transfer resources from the central level to the school ..., and then 
how do we provide accountability of those resources—all those things ... 20 years 
later are well known processes, not only in the Bank but in the United Nation’s 
Children’s Fund, USAID, the Japan International Cooperation Agency—we know 
about those processes. Twenty years ago, they evolved from El Salvador, and they 
have become now part of international knowledge that some of our younger staff, the 
newer countries that are developing community participation programs don’t know 
that that evolved from El Salvador. My staff worked with Afghanistan, you know, 


21 These three ministers were: Abigail de Perez (Minister during 1998-1999, formerly Vice-Minister and 
Director of Planning for the MINED), Evelyn Jacir de Lovo (Minister during 1999-2003, formerly Director 
of the Modernization Unit within the MINED, later working for the Organization of American States as the 
Director of the Departamento de Desarrollo Social y Empleo), and Darlyn Meza (Minister during 2004-2009, 
formerly Vice-Minister, Director of Education, and Coordinator of the EDUCO office). Only Rolando 
Marin, who briefly served served as Minister during 2004, did not have a previous association with the 
EDUCO reform. 
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they don’t know. The ministry of education didn’t know that what they call 
community participation schools and all these operational processes, that, you know, 
in organizational management, in financial transfer, in planning, that it evolved from 
this tiny nation. (INTACT34) 

Fifth, by imbuing EDUCO with an aura of success, these studies have created a foundation of 
legitimacy that allows the World Bank to call on the former champions of the program for support. 
According to Edwards (2013b), one key functionary, who worked with the EDUCO program for 
many years early on mentioned that she would eventually “know all of Latin America ... and a great 
part of the world because of EDUCO” - about 25 to 30 countries in all (p. 247). This functionary 
goes on to say, “people wanted to know how it was possible that illiterate rural people could 
manage, could hire, could fire teachers, how it was that the poor could be interested in school. That 
is was people had a hard time understanding” (Edwards, 2013b, pps. 247-248). More recently, in 
2011, this same interviewee was sent by the World Bank to Afghanistan to discuss the EDUCO 
model. For the World Bank, such Salvadoran education specialists have played a key role in 
EDUCO’s promotion. In the words of the above-mentioned interviewee, “I helped a lot with the 
community theme. When people [(i.e., governments)] were scared to give money to the parents, I 
helped to make the school boards and transfer the funds to the schools ... in many places” 
(Edwards, 2013b, p. 248). This former Salvadoran education specialist served as reinforcement when 
officials in other countries were reluctant to experiment with the EDUCO model. Elere again, 
actions such as being called on to defend and spread the program would not have been possible—or 
arguably would not have occurred to the same extent—without the track record of success 
outcomes that had been established through the studies discussed in this paper. 

The sixth, final, and perhaps most direct international implication is one that still has 
relevance: the evidence base discussed here has become a body of literature that exists independently 
of the EDUCO program. Appendix 3 contains extensive examples. In terms of the World Bank, 
staff there continue to make use of the evidence base produced during the height of attention to 
EDUCO. In more recent publications, as they review and write on findings related to 
decentralization in practice, they draw on EDUCO studies, despite their limitations, and in so doing 
signal what is, from their perspective, the enduringly positive and strong example of EDUCO (see, 
e.g., Barrera-Osorio, Fasih, Patrinos, & Santibanez, 2009; Bruns, Filmer, & Patrinos, 2011; Di 
Gropello, 2006, 2007; Umansky & Vegas, 2007). As with the World Bank, so too do other 
institutions continue to draw on the EDUCO studies reviewed here. Although this is evident from 
the list found in the introduction and in Appendix 3, included here are a few key examples (apart 
from the World Development Report 2004, discussed above) that demonstrate that institutions as 
diverse as the Brookings Center, the Global Partnership for Education, and UNESCO have recently 
found the lessons highlighted in EDUCO evaluations to be relevant. 

• Brookings Institution. 2008 : A report on improving development effectiveness spotlights 
EDUCO and the fact that the school council “hires, monitors, retains or dismisses teachers” 
(Flartman & Linn, 2008, p. 47). 

• World Bank. 2009 : A World Bank review of school-based management highlighted and 
classified the EDUCO program as one of the stronger models because community members 
are responsible for “hiring, firing and monitoring teachers” (Barrera-Osorio, Fasih, Patrinos, 
& Santibanez, 2009, p. 7). 




El Erograma EDUCO 


27 


• Global Partnership for Education. 2009 : The Global Partnership for Education (2009) 
focused on EDUCO as a “way to ensure that the community contributes to improved 
educational outcomes” (p. 96). 

• UNESCO. 2009 : In the Global Monitoring Report, UNESCO underscored that EDUCO is 
a model that increases client “voice,” meaning that it entails mechanisms through which 
community members have recourse if teachers do not meet their expectations (UNESCO, 
2008). 

• World Bank. 2011 : In a review of evidence on accountability mechanisms in education, the 
World Bank again focused on the EDUCO program as one of the “stronger” examples 
(Bruns, Filmer, & Patrinos, 2011). 

• World Bank. 2012 : A typical example is found in a book written by World Bank specialists 
about education reforms for sub-Saharan Africa. They state: “the authority of community- 
managed EDUCO schools to hire and fire teachers in El Salvador had a beneficial impact on 
student outcomes compared with traditional schools serving similar populations” (Majgaard 
& Mingat, 2012, p. 149). 

• Save the Children. 2013 : In a report on community participation, EDUCO is cited in 
relation to what mothers can do to enhance student outcomes (Save the Children, 2013). 

It should be noted, though, that the above mentioned institutions have highlighted EDUCO for a 
variety of other reasons that do not have anything to do with the original goals of the program. In 
looking at the evidence on EDUCO, they have found—or have chosen to see—lessons related not 
only to instituting community involvement in education (Global Partnership for Education, 2009), 
but also to extending system coverage rapidly (UNESCO, n.d.), achieving educational and gender 
equality (Birdsall, Levine, & Ibrahim, 2005), providing education in post-conflict contexts (Beleli et 
al., 2007; UNESCO, 2011), and ensuring the provision of education as a human right (UNESCO, 
2007). In these ways, one sees, on one hand, how the evidence base created by the World Bank 
continues to echo throughout the international development landscape, and, on the other hand, how 
various institutions have adapted the lessons of EDUCO to their own purposes. In other words, the 
evidence on EDUCO has become symbolic; these studies have a reputation for being rigorous, or 
are often otherwise perceived by their readers as such, and so development professionals invoke 
them as a way lend credibility to the points they wish to advance. Going forward, this will likely be 
one of the primary legacies of the EDUCO program internationally, especially as development 
trends change, and as community-level decentralization becomes less popular, replaced by such hot 
topics as public-private partnerships (Patrinos, Barrera-Osorio, & Guaqueta, 2009). 


Conclusion: Limitations and Future Possibilities 


The present manuscript began by examining the historical context within which EDUCO 
emerged in order to explain the dynamics that surrounded the program from the outset. This was 
the necessary starting point because the studies reviewed here were, in addition to being produced 
by and for institutions and individuals with vested interests, also borne into a very receptive political- 
economic context, that is, a context in which notions of decentralization were already popular and in 
which the idea of community management of education was received with excitement because it 
represented the next step in experimentation around transferring responsibilities away from the 
central ministries of the State. Subsequent sections of the article then engaged in a detailed re- 
evaluation of the widely-known and cited studies on EDUCO, followed by a consideration of 
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what—in light of this re-evaluation—can reasonably be claimed to be known about the EDUCO 
program. Finally, a discussion was presented of the various implications of these EDUCO studies 
and the ways that they intersected with the political-economic context of the 1990s and 2000s. 

In this review, it has not only been demonstrated that the evidence used to promote 
EDUCO is fundamentally flawed, but also that one cannot be sure of any of the outcomes claimed 
by the studies reviewed here. Moreover, as opposed to attributing the observed outcomes to the 
EDUCO program, it was showed that there is reason to believe that the outcomes could be the 
result of pre-existing social capital. Recall that the EDUCO program was initially targeted to 
communities which were affiliated with the FMLN and which, as such, had experience with various 
forms of self-organization, including the system of popular education schools that had developed 
during the civil war. Given these experiences, it may be that the EDUCO communities were well 
prepared to engage in the kind of community-level management that the EDUCO program entailed. 
Put differently, there is reason to believe that the EDUCO communities included in the studies 
reviewed here had relatively more social capital than the communities surrounding the traditional 
mral public schools. To the extent that this is the case, the control group is dissimilar and cannot be 
compared to treatment group of EDUCO communities for the purpose of determining the impact 
of the EDUCO program. 

Returning to the studies reviewed in this study, they are generally problematic for two 
reasons that go beyond their specific findings. The first is the econometric nature of the studies, 
while the second is the political-financial-intellectual complex from which these studies were borne 
and back into which they went as they furthered the interests of that complex. On the first point, 
one must note that econometric methods, while useful for indicating the general nature of 
relationships or correlations among variables, cannot by their nature offer causal explanations. On 
one hand, this is because data do not speak for themselves, and so researchers and practitioners 
must interpret the relationships among variables based on their guiding frameworks, whether they 
be implicit or explicit (Kvernbekk, 2013). On the other hand, this is because econometric studies 
look at inputs and outputs without being able to unpack the “black box” of how a policy works 
(Pawson, 2006). That is to say, econometric methods are limited in their ability to explain how and 
why each sub-component of a program—such as EDUCO, for example—works or does not work in 
practice. These methods restrict their focus to what goes in and what comes out, thereby failing to 
account for both the inner-workings of a program as well as the many layers of context that impinge 
on how the program operates in implementation (Pawson & Tilly, 1997). In the words of Dumas 
and Anderson (2014), “Achieving high levels of rigor in quantitative studies involves limitations that 
include social and cultural decontextualization, ahistoricity, [and] the creation of randomization that 
is seldom generalizable to real life settings” (p. 5). A fundamental point is that, while quantitative 
data are considered ideal for empirically justifying the implementation of education policies (that 
ultimately have and always had political motivations), these data are not sufficient. 

It is for the above reasons that in-depth, thorough, and analytic qualitative studies are 
necessary for answering policy-relevant questions and for providing policy-relevant findings. While 
Dumas and Anderson (2014) discuss a range of possibilities, a few approaches are highlighted here 
that hold promise for informing policy. Given space constraints, it is only possible to mention these 
approaches briefly; this is done with the hope that they may spark interest among researchers as 
useful ways to investigate complex educational issues. These approaches are vertical/horizontal case 
studies (Vavrus & Bartlett, 2006), process tracing (Bennett & Elman, 2006), realist case study 
(Pawson, 2006), systems thinking (Gillies 2010), ethnographic evaluation (Whitehead, 2002), 
participatory evaluation methods (Upshur, 1995; World Bank, 2011), and narrative research 
(sometimes also known as life history research) (Lieblich, Tuval-Mashiach, & Zilber, 1998; Moen, 
2006). The common characteristic that these approaches share is that they that complement and 
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expand the findings that econometric analyses can offer, for example, by illuminating the lived 
experience and emic perspectives of stakeholders (narrative research, participatory project 
assessment), by revealing which policy mechanisms work in which contexts (realist evaluation), by 
indicating how policies are implemented and affect the daily lives of people in practice (ethnographic 
evaluation, participatory action research), by providing insight into how the same policy can be 
interpreted and implemented differently across multiple levels and multiple locations 
(vertical/horizontal case study), by showing various sub-systems and various aspects of the 
education system impact one another (systems thinking), and by illustrating how multiple actors and 
multiple events influence one another in the process of policy making and policy implementation 
(process tracing). As can be seen, through these methods, researchers are frequently able to arrive at 
more nuanced and more explanatory findings than would be the case through the exclusive use of 
econometric methods. 

Going back to the second issue mentioned above—i.e., the kind of political-financial- 
intellectual complex that tends to produce impact evaluations and the consultants who carry them 
out—the work of Samoff (1993) is relevant. As a scholar who focuses on African education reform 
in the context of international development efforts, and often under external pressure, Samoff 
(1993) has witnessed conditions similar to those which engulfed El Salvador in the late 1980s and 
1990s. He characterizes the dynamics of reform in such contexts in this way: 

Having concluded that local decision makers have fundamentally mis-managed their 
responsibilities, the external agencies offer general and rapidly disbursed support in 
exchange for broad control. This broader form of external assistance impels a 
demand for broader expert knowledge. The expertise required goes well beyond, say, 
curriculum development and teacher training. The relevant experts are those who 
can understand and manage production, finance, international exchanges, and 
national planning as well as social services. This call for broader expertise comes at a 
moment of severe economic crisis, precisely the time when African countries are 
least able to supply expertise and experts at the level and scale and with the 
credibility and legitimacy the external agencies demand. (Samoff, 1993, p. 186) 

This description is reminiscent of the context and sequence of events that President Cristiani 
experienced after being elected in 1989. Not only was El Salvador subject to external pressure from 
USAID and the World Bank to enact certain reforms for both the economic and social sectors, but 
Salvadoran entities such as FUSADES, the MINED and the Ministry of health also had to work 
with international consultants and researchers from UNESCO and the World Bank, among other 
foreign institutions, who possessed the necessary clout and technical capacity to credibly guide the 
reform and to carry out the research that would be used for decision-making around the reform of 
social services, such as education. 

But Samoff (1993) makes an additional observation about the relationship between national 
and international actors that is key for our purposes. In his words: “what is most powerful and most 
insidious in this relationship is the internalization within Africa of worldviews, research approaches, 
and procedures for creating and validating knowledge that effectively perpetuate Africa's 
dependence and poverty” (Samoff, 1993, p. 187). The point here is that, over time, certain 
approaches to research—typically the econometric and advanced statistical approaches utilized by 
the consultants—are seen as the only or the most legitimate methods of investigation, particularly 
for the purpose of policymaking (Samoff, 1996). The implication in practice is that local researchers 
strive to replicate technically complex methods, in order to for their research to achieve the same 
legitimacy, though often researchers in middle and low-income countries are hindered by a lack of 
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training in these methods and by a lack of funding to carry out costly studies. One consequence of 
this situation is that local researchers continue to rely on international consultants and international 
organizations for both financial and technical resources, in addition to relying on them for their 
reputation and status as capable knowledge producers. A second consequence is that qualitative 
methods, such as those mentioned above, are seen as less desirable and less appropriate when it 
comes to their ability to inform education sector reform, despite the fact that quantitative methods 
suffer from numerous shortcomings themselves (Erickson & Gutierrez, 2002; Klees & Edwards, 
2014; Learner, 1983; Samoff, 1991, 1996; Quade, 1970; Schroeder, Sjoquist, & Stephan, 1986; 
Verschuren, 2001), though they are often unrecognized or purposefully overlooked, as well as the 
fact that the relevance of qualitative research to policymaking has been thoroughly defended 
(Dumas & Anderson, 2014). 

Both of the above-noted consequences have certainly played out—and continue to play 
out—in El Salvador. Going forward, it will be necessary to work in a number of different areas to 
address this situation, especially to the extent that the goal is to ensure that there are well-trained 
Salvadoran researchers who can work within and outside the Salvadoran government to produce 
policy-relevant research from numerous theoretical perspectives and through various research 
methods. Based on this case from El Salvador, it is also suggested that an additional goal should be 
to incorporate and to prioritize research that is conducted through the lens of social justice. A few 
broad, interrelated strategies to work towards these ends include: first, reducing the power imbalance 
that has allowed external actors and external studies to hold such influence; second, seeking alliances 
with domestic and international organizations who themselves prioritize principles of social justice 
in their research studies and in their policy recommendations; third, developing research centers and 
the research capacity necessary to counter mainstream (often neoliberal) studies and rhetoric around 
policy; fourth, connecting these centers and local researchers with international think tanks, 
international scholars, transnational social movements, and networks of activists in order to provide 
alternative and/or novel perspectives on currendy relevant issues. These strategies would also help 
to counter, where necessary, the policy messages that are being promoted around the world and in 
El Salvador by dominant (often neoliberal) knowledge institutions. The importance of these 
suggestions cannot be understated, as making progress in these areas would drastically transform 
how education policies are made (and how, or whether, they go global), particularly in low- and 
middle-income countries, and particularly given the entrenched nature of neoliberal frameworks 
used for understanding and for reshaping the provision of social services. Importantly, as has been 
highlighted in this paper with regard to the World Bank, these frameworks underlie the methods 
used and shape the findings and recommendations offered, with the implication being that there is 
an ideological agenda within the World Bank that is not only protected from research findings but 
which itself informs the findings produced through ostensibly objective research. In the worst case 
scenario, World Bank researchers use empirical data to justify those policies that fit with the 
organization’s preferences (Broad, 2006; Verger, 2012). At the global level, one indicator of the 
timeliness of the suggestion to increase the research capacity of southern knowledge agents is the 
fact that it has been proposed by actors involved in the process of setting the post-2015 goals (Quint 
& Lucas, 2014). 

In the end, it is hoped that the present paper will help to take a step forward to address the 
issues highlighted in this concluding section. As has ben argued, when it comes to research on 
education policy, new methods and new political-institutional-financial arrangements are needed, a 
point which the present review of the impact evaluations of EDUCO has made clear. 
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Appendix 1 

Summary of the Findings and Limitations of the World Bank’s Evaluative 

Studies of the EDUCO Program 


No. 

Study 

Methods. 

Data* 

Findings Offered 

Limitations/Comment 

1 

World 

Bank 

(1994) 

Two-group, 
ex post 
comparison 
and 

regression 
analysis, 
both with 
1993 data 
from 78 
communities 

- EDUCO students in 
grades 1 and 2 show 
better results on math 
exam in two-group 
comparison, but not in 
regression analysis 

- Without statistical controls, it is not 
possible to determine if the differences 
observed in the two-group comparison 
are due to the EDUCO program. 

- In terms of regression, which the 
authors only used to predict preschool 
achievement outcomes, the only 
significant variable relates to family 
income (larger income = higher 
achievement); the EDUCO variable did 
not show an impact 

2 

Umanzor 
et al. 

(1997) 

Two-group, 
ex- post 
comparison 
with 1996 
data 

- ACE members feel 
that they have more 
influence in schools 
decisions (although 
they do not make 
more decisions in 
practice) 

- ACE members visit 
schools more often 

- EDUCO teachers 
meet/contact parents 
more often 

- In terms of academic achievement, the 
authors conclude that EDUCO schools 
are no worse than traditional public 
schools, even when their scores are 
lower, because student learning scores 
not statistically significantly worse 

- Among the studies reviewed here, this 
is the one with the most credibility 

3 

Jimenez 

& 

Sawada 

(1999) 

Regression 

(OLS) 

analysis with 

Heckman 

two-step 

correction 

for sample 

selection 

bias with 

1996 data 

- Effect of EDUCO 
on language scores is 
positive & significant 

- The more fully specified models are not 
significant; the authors decide to focus 
on the model with the fewest controls 
but with significant results. 

- Endogeneity: EDUCO students possess 
unobserved characteristics that correlate 
negatively with achievement outcomes; 
cannot parse out tme impact of 
intervention 




-EDUCO 
significantly reduces 
student absences 

- Reduction in absences could have been 
the result of the timing of new school 
construction/new educational services 
offered (shortly before data collection in 
1996) and not a result of the EDUCO 
program itself (recall that many of these 
communities initially lacked schools) 
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Appendix 1 cont’d 

Summary of the Findings and Limitadons of the World Bank’s Evaluadve 

Studies of the EDUCO Program 


No. 

Study 

Methods. 

Data* 

Findings Offered 

-© - 

Limitations / Comment 

4 

Sawada 

(2000) 

Regression 

(OLS) 

analysis with 
instrumental 
variables 
with 1996 
data 

- Community 
participation positively 
and significantly 
related to teacher 
wages 

- Not the case in more fully specified 
model-Moreover, in models where 
variables for (a) community participation 
and (b) the presence of an EDUCO 
school are significant, the latter is 
negatively related to teacher salary. In 
other words, the presence of an EDUCO 
school corresponded with lower teacher 
salaries 

- Community 
participation (i.e., ACE 
member school visits) 
enhances teacher 
effort (i.e., meetings) 
with parents 

- Simultaneity; not possible to separate 
effects or directionality of influence 
between variables. That is, both variables 
(participation and teacher effort) are 
inter-related and, as such, cannot indicate 
a concrete and independent relationship 
with the dependent variable 

- Community 
participation in 

EDUCO associated 
with lower student- 
teacher ratio 

- Spurious relationship; EDUCO schools 
in sample have fewer students. 

-EDUCO 

participation positively 
and significantly 
effects student 
achievement 
- EDUCO resulta 
logros academicos 
mejores (en terminos 
estadisticos) 

- The participation variable with an effect 
is not specific to EDUCO, but represents 
participation in by parents in councils of 
traditional and/or EDUCO schools— 
and, as such, we do not know if the 
observed effect is due to the EDUCO 
program 

5 

Jimenez 

& 

Sawada 

(2003) 

Regression 

(probit) with 

Heckman 

two-step 

correction 

with 1998 

data 

- Attending EDUCO 
increases probability 
of remaining in school 
during grades 4-6 

- Endogeneity: the unobserved 
characteristics that affect the likelihood 
of participating in EDUCO also 
negatively effect the decision to stay in 
school 
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Appendix 1 cont’d 

Summary of the Findings and Limitadons of the World Bank’s Evaluative 

Studies of the EDUCO Program 


No. 

Study 

Methods, 

Data* 

Findings Offered 

-©- 

Limitations / Comment 

6 

Sawada 

& 

Ragatz 

(2005) 

Regression 
(OLS) 
with PSM 
and 

Heckman 
two-step 
correction 
with 1998 
data 

- EDUCO schools 
produce more frequent 
community 
participation (i.e., 
teacher meetings with 
parents) and teacher 
effort (i.e., teaching 
hours), and fewer 
teacher absences. 

- 76 models estimated, variables 
inconsistently significant 




- EDUCO leads to 
increased language 
scores 

- Additional controls were available which 
were not used in the models presented with 
statistically significant variables; when 
applied, language scores were no longer 
significant 

Key: EDUCO = 
propensity score 

Education with Community Participation Program; OLS = ordinary least squares; PSM = 
matching 

Notes: *Data from 1993 come from a sample of 78 schools. This sample include 33 sections of preschool and 

45 sections of grades one and two. In each section for grades and and two, 17 students took the language and 
math tests. Basic information was also collected about the family to use as controls. Data from 1996 is based 
on random sample of 311 schools with a third grade section of at least 10 students. The surveys administered 
included language and math tests, in addition to a household survey and questionnaires for the director, third- 
grade teacher, parents, and two ACE members. Data from 1998 added new student learning tests and teacher 
and parent surveys to the information collected in 1996. The comparison group for all quantitative studies 
was traditional rural public schools. 
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Appendix 2 

The Global Promotion of EDUCO, 1996-2006 

• 1996 — A UNESCO Bulletin on meeting the EFA goals in Latin America mentions the EDUCO 
program as an example of decision-making power being transferred to the local level 
(UNESCO, 1996). 

• 1997 — A regional seminar for Central America is financed by the World Bank to disseminate the 
lessons of EDUCO (Meza, 1997). 

• 1997 — EDUCO wins the President’s Award for Excellence from the World Bank (Eriksson, 
Kreimer, & Arnold, 2000). 

• 1997 - EDUCO is labeled a flagship program by the World Bank (Eriksson, Kreimer, & Arnold, 

2000 ). 

• 1998 - The IDB features EDUCO at its 1998 meeting of the board of governors in Cartagena, 
Colombia (IDB, 1998). 

• 1998 - The Revista Latinoamericana de Innovaciones Educativas features the findings on EDUCO of 
the special mission of the IDB to El Salvador (Merino, Ycardo, & Jacir de Lovo, 1999). 

• 1999 - EDUCO is identified in a regional UNESCO Bulletin as a strategy for providing quality 
education to underprivileged and/or excluded youth (Rivero, 1999). 

• 1999 - A special issue of the World Bank Economic Review on decentralization and voucher 
experiments in education features the research of Jimenez and Sawada (1999) on the EDUCO 
program (World Bank, 1999). 

• 2000 — The World Bank commissions the creation of teaching materials on the EDUCO 
program. Edge (2000) creates a “resource-kit” on EDUCO. 

• 2000 - In the IDB’s Primary and Secondary Education Strategy, EDUCO is mentioned as an 
exemplar of reform based in local management and accountability. It is claimed that “research 
has shown that school attendance and student achievement improved significantly as a result of 
these reforms” (IDB, 2000, p. 19). IDB (2000) additionally asserts that “community monitoring 
can also help to reduce absenteeism by putting community pressure on offending teachers” and 
that “El Salvador’s EDUCO program has been particularly effective in these approaches” (p. 

19). 

• 2000 - In a book published by IDB, a former Minister of Education from El Salvador 
contributes a chapter on the benefits of community participation (Jacir de Lovo, 2000).“ 


22 The same minister of education would later also contribute a chapter on EDUCO in a book published by a 
Salvadoran think tank (Jacir de Lovo, 2003). 



El Erograma EDUCO 


43 


Appendix 2 cont’d 

The Global Promodon of EDUCO, 1996-2006 

• 2001 - At an international workshop in Lima, Peru, co-financed by the World Bank and titled 
the “International Workshop on Participation and Empowerment for Inclusive Development,” 
the case of EDUCO is highlighted by the World Bank (Meza, 2001). 

• 2001 - Through a book chapter UNESCO reviews the gains made by El Salvador in the 1990s 
and exclaims its desirability for increasing community participation (UNESCO, 2001). 

• 2001 — A conservative education think tank — the Partnership for Education Revitalization in the 
Americas (PREAL) — led by the former Minister of Education from El Salvador focuses on the 
potential of EDUCO to address issues of educational coverage (PREAL, 2001). 

• 2003 - At an international workshop sponsored by the Organization of American States (OAS) 
to promote knowledge sharing among countries, a Salvadoran MINED representative presents 
on EDUCO (Najarro, 2003) 

• 2004 - The World Bank features the gains of the EDUCO program in its En Breve publication 
(Meza, Guzman, & de Varela, 2004b). 

• 2004 - World Bank consultants from El Salvador present on the successes of the EDUCO 
program at a conference in Shanghai, China themed “Reducing Poverty on a Global Scale: 
Learning and Innovating for Development” (Meza, Guzman, & de Varela, 2004a). A book is 
produced by the World Bank based on this conference, in which EDUCO’s virtues are again 
noted (Moreno-Dodson, 2005). 

• 2004 - In its flagship publication, the World Development Report, the World Bank spotlights 
EDUCO as the prime example of public service decentralization in education along lines of 
accountability and for reasons of efficiency and effectiveness (World Bank, 2003). Per the Bank’s 
own admission (World Bank, 201 lb), this report has been tremendously impactful, both within 
and beyond the Bank. Within the Bank, its regional strategies and programmatic activities at the 
country level were re-aligned in terms of WDR 2004’s accountability framework. Externally, the 
World Bank has seen the influence of WDR 2004 on DFID’s work. 

• 2005 - In a book produced by the World Bank on revising service delivery, EDUCO is singled 
out as a reform that engendered access, efficiency, and equity (Fiszbein, 2005). 

• 2006 - The publication En Breve of the World Bank again covers the EDUCO program, this 
time to speak to issues of access and quality (World Bank, 2006). 

• 2006 - The World Bank suggests to China at a conference in Peking on successful development 
interventions that it base its approach to mral education on the EDUCO model (Ramirez, 2006). 
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Appendix 3 

Examples of EDUCO in Global Literature 

• 2005 - A United Nations Task Force on Education and Gender Equality identified EDUCO as 
“probably the most celebrated case of successful parental control,” as well as a key strategy for 
improving primary school effectiveness (Birdsall, Levine, & Ibrahim, 2005, p. 67) 

• 2006 - The evidence base for EDUCO is drawn upon in a World Bank report which elevates 
and compares educational decentralization in four Central American countries, suggesting them 
as good examples of teacher accountability (Di Gropello, 2006; see also Di Gropello 2005). 

• 2007 - A book chapter by a World Bank education specialist on forms of educational 
decentralization in Latin America includes EDUCO (Di Gropello, 2007). 

• 2007 - A report prepared for UNICEF suggests that the EDUCO experience can provide 
lessons for decentralization in emergency/post-conflict settings in Africa (Beleli et al., 2007) 

• 2007 - In a UNESCO (2007) report on how decentralization can help to ensure the provision of 
education as a human right, the EDUCO program is featured. 

• 2007 — An article on decentralization in three Central American countries by World Bank 
researchers reviews the group 1 studies and signals that such reforms lead to improvements in 
student learning and teacher acountability (Umansky & Vegas, 2007). 

• 2008 — A report by the Brookings Center on improving development effectiveness spotlights 
EDUCO as “an example of how a small innovation can be scaled up to a national level and have 
a significant impact on national education systems” (Hartman & Linn, 2008, p. 47) 

• 2009 - The Global Partnership for Education (2009) focused on EDUCO as a “way to ensure 
that the community contributes to improved educational outcomes” (p. 96). 

• 2009 - A World Bank review of experience with decentralization (or, in this case, “school-based 
management”) underscored the EDUCO program and rehearsed by the benefits claimed by 
group 1 studies (Barrera-Osorio, Fasih, Patrinos, & Santibanez, 2009). 

• 2011 - In a review of evidence on accountability mechanisms in education, the World Bank 
again focuses on the EDUCO program as one of the “stronger” examples (Bruns, Filmer, & 
Patrinos, 2011). 

• 2011 - The Global Monitoring Report on armed conflict and education suggests that EDUCO 
provides a good example of a course of action when governments fail to provide educational 
services (UNESCO, 2011). 

• 2012 - A website for exemplary educational innovations in Latin American and the Caribbean 
features the EDUCO program (UNESCO, n.d.). 
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Appendix 3 cont’d 

Examples of EDUCO in Global Literature 

• 2012 - A book written by World Bank specialists about education reforms in sub-Saharan Africa 
states that “the authority of community-managed EDUCO schools to hire and fire teachers in 
El Salvador had a beneficial impact on student outcomes compared with traditional schools 
serving similar populations” (Majgaard & Mingat, 2012, p. 149). 

• 2013 - In a report on community participation, EDUCO is cited by the Save the Children 
organization in relation to what mothers can do to enhance student outcomes (Save the 
Children, 2013). 
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